{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 逻辑回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 什么是逻辑回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "逻辑回归（Logistic Regression）也是机器学习一个最基本也是最常用的算法模型。与线性回归不同的是，逻辑回归主要用于对样本进行分类。因此，逻辑回归的输出是离散值。对于二分类问题，通常我们令正类输出为1，负类输出为0。例如一个心脏病预测的问题：根据患者的年龄、血压、体重等信息，来预测患者是否会有心脏病，这就是典型的逻辑回归问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "二元分类，一般情况下，理想的目标函数$f(x)\\geq0.5$，则判断为正类1；若$f(x)<0.5$，则判断为负类0。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- $f(x)\\geq0.5$：$\\hat y=1$\n",
    "\n",
    "- $f(x)<0.5$：$\\hat y=0$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 得分函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "与线性回归类似，令逻辑回归的输入为x，维度为（m，k）。其中，m为样本个数，k为输入特征维度。输出为y，维度为（m，1）。引入参数$w_0,w_1,\\cdots,w_{k}$，计算线性回归的得分函数（score function）为："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$s=w_0x_0+w_1x_1+\\cdots+w_kx_k$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "但是这里得到的$s$是连续输出，范围为整个实数，无法应用与分类问题。方法是对得分函数$s$再处理，利用Sigmoid函数将数值限定在[0,1]之间。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Sigmoid函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sigmoid函数的表达式为："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\\theta(s)=\\frac{1}{1+e^{-s}}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "相应的函数图形为："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8FPX9x/HXl0ASkHATOUIIyBmu\nAAHEAw9AkapB0QoiiqCUVqW1tQ+1nvVoPaqorRdVRFCBigd4IoioyCEBAiEcIYQjCVcSIBBzb76/\nP7L4WzGQhWyy1/v5eOTB7sx3Zz6Z3byZ/c53Zoy1FhERCSx1vF2AiIh4nsJdRCQAKdxFRAKQwl1E\nJAAp3EVEApDCXUQkACncRUQCkMJdRCQAKdxFRAJQXW+tuEWLFjYmJsZbqxcR8Utr167Nsda2rKqd\n18I9JiaGxMREb61eRMQvGWN2u9NO3TIiIgFI4S4iEoAU7iIiAchrfe6VKS0tJTMzk6KiIm+XIqcQ\nHh5OVFQU9erV83YpInISPhXumZmZREREEBMTgzHG2+VIJay15ObmkpmZSYcOHbxdjoichE91yxQV\nFdG8eXMFuw8zxtC8eXN9uxLxcT4V7oCC3Q/oPRLxfT4X7iIigcpay0tfb2fz3qM1vi6f6nMXEQlU\n5eWWRz9JYdbK3RSUOIht06hG16c990q8/vrr3HHHHdVaxpIlSxg/fvwZv76wsJCLLroIh8PBxIkT\niYyMpGfPntWq6VQyMjK4/fbbueeee1iyZAklJSUMGTKEsrKyGlunSLAoc5Rzz/wNzFq5m8lDOnLv\niK41vk6FeyU2btxIr169qrWMDRs20KdPnzN+/YwZM7j22msJCQlhwoQJfPnll269btmyZUyYMOG0\n17d161ZCQ0OZOnUqw4YNIzQ0lKFDhzJv3rzTXpaI/L/iMgd3zVnPh+uy+PPwLtx/RbdaOW6lcK9E\ncnKyR8I9Li4OqAjOIUOG0KNHD4YNG0ZOTg4AW7ZsYciQIfTu3Ztnn32WTp06/fz6d999l4SEBACG\nDBlCs2bNqlXPcTt37iQhIYH4+HgGDhzItm3bABg+fDh33XUXd955J1lZWQCMGjWKd9991yPrFQlG\nhSUObp+1li827eehK2OZOrRzrQ1IULhXYtOmTSftArnwwguJi4v71c+SJUt+0e74nntxcTGjR4/m\nxRdfJCUlheHDhzNt2jTKysoYN24cL774Ihs3biQ9Pf3ndZaUlJCeno6nr5pZWlrKbbfdxvPPP09i\nYiKPPvooTz31FAD33nsvb7zxBtHR0URGRgLQs2dP1qxZ49EaRILF0aJSbp6xmuXbs3lmdG8mXVC7\n54X47AHVv3+S4vEjyrFtGvHIVT1O2SYjI4OIiAgaN25c6fzvv/++yvWUlpZy9OhRWrZsybx587jg\nggvo27dvRQ2xsSxcuJAPP/yQPn36/GL68VDNycmhSZMmp/OrMWjQIIqLi8nPz+fQoUM/f2t4+umn\nufzyywH4+OOPSUlJYfTo0QCUlZVx4YUX/tzuRCEhIYSGhnLs2DEiIiJOqx6RYJabX8wtb/3I1n3H\neGlsX67s3abWa/DZcPeWqvrbL7zwQo4dO/ar6f/6178YNmwYAJs3b6Z79+4/P3ZdXnJyMrGxsWzc\nuPHnAIaKbwsjRowAoH79+qd9ktDq1auBij73mTNnMnPmzF+12bBhA08++SSTJk1ye7nFxcWEh4ef\nVi0iwWxfXiE3vbGazMOF/PfmeC7pFumVOnw23Kvaw64pVfW3u7Pn7trf3rZtW5KSkgBIT09n9uzZ\nLF++nFmzZpGamgpAUlIS77zzDvfeey8ATZs2xeFwUFRU5NFgbd26NYsWLeLWW2+lTp06JCcn07Nn\nz5P2Aebm5tKyZUtdQ0bETbtyfmLcG6vJKyxl1sSBDOrY3Gu1qM/9BMnJyUyfPp2YmBhiYmIYPHjw\naS/DdaTM+PHj2bt3L7169WLMmDHMmDGD5s2bM378eBITExkwYAAzZswgJiaGjh07/ryMyy67jOXL\nlwMwduxYBg8ezLZt24iKiuLNN988o99t4sSJlJeX0717d+Li4nj66adPeXDnm2++YeTIkWe0LpFg\ns2XfUa57bSWFpQ7m3H6uV4MdwFhrvbLi+Ph4e+KdmLZs2fJzd0agy8/Pp2HDhgA8++yz5OXl8cQT\nT/w8f/369Tz//PPMnj3bWyVy7bXX8s9//pOuXX89JjeY3iuRqqzdfYhb31rDWWF1mT1pIJ0ia+4Y\nlTFmrbU2vqp22nP3kmnTptGjRw/i4uLYtWsXDz300C/m9+3bl0suuQSHw+GV+kpKShg1alSlwS4i\n/+/b1GzGvbGa5g3DeH/K4BoN9tOhPXc5I3qvROCzjfv407z1dIqMYNbEgbSMCKvxdbq75+6zB1RF\nRHzZe6v38MDHyfSPbsqbEwbQuL5vDTxQuIuInAZrLa9+u4NnvtzGJV1b8sq4/tQPDfF2Wb+icBcR\ncZO1ln9+sZXp36WTENeGf13fh3ohvnnoUuEuIuKGMkc5932YzPy1mdw8uD2PXtWDOnV898Y1Phfu\n1lrd6cfHeesgvIi3FJU6uPO99SzZcoA/DevMH2vxAmBnyqfCPTw8nNzcXN1H1Ycdv0G2LkkgwSKv\nsJTbZyWyZtchHkvowc2DY7xdklt8KtyjoqLIzMwkOzvb26XIKYSHhxMVFeXtMkRq3MGjRdw840d2\nZOfzwg1xJMS19XZJbqsy3I0xM4ArgYPW2l9dB9cYMw641/k0H/i9tXbDmRRTr149OnSo3ctiiohU\nZnfuT4x/80dy8ot585YBDOnS0tslnRZ3DvPOBEacYv5O4CJrbW/gcWC6B+oSEfGaTVl5jH51BceK\nSnn3tkF+F+zgxp67tfY7Y0zMKeavcHm6CtD3dRHxWyt25DB51loa16/H2xMH0imyobdLOiOe7nOf\nBHzh4WWKiNSKL5L38ce5ScS0aMDbEwfSunF9b5d0xjwW7saYS6gI9wtO0WYyMBkgOjraU6sWEam2\n2St38fDCFPpFN+XNW+Jp0iDU2yVVi0dOrTLG9AbeABKstbkna2etnW6tjbfWxrds6X99WCISeKy1\nPP/VNh5akMLQbpG8M2mQ3wc7eGDP3RgTDXwIjLfWpla/JBGR2lHmKOehBZuY82MGN8S348lrelLX\nRy8ncLrcGQo5B7gYaGGMyQQeAeoBWGtfAx4GmgOvOE88KnPncpQiIt5UWOLgrjkVZ53eeUkn/nJZ\nl4A6edKd0TJjq5h/G3CbxyoSEalhRwpKmPR2Iuv2HObxhB6M95OzTk+HT52hKiJS0zIPFzDhrTXs\nyS3glRv7cUWv1t4uqUYo3EUkaGzee5QJb/1IYamDWZMGcq6Xb2JdkxTuIhIUVqTl8LvZa2kYXpf5\nU86jayvfuNdpTVG4i0jAW5CUxT3vb6BDi7OYeetA2jTx35OT3KVwF5GAZa1l+nfp/POLrQzq0Izp\nN8f73L1Oa4rCXUQCkqPc8vinm5m5Yhe/6d2a53/bh7C6vnev05qicBeRgFNU6uCPc9ezKOUAky7o\nwAMju/v0LfFqgsJdRALKoZ9KuO3tNazPOMJDV8Yy6YLgvEeEwl1EAsbu3J+49a01ZB4pDOgx7O5Q\nuItIQFi/5zC3vZ2Iw1reu20Q8THNvF2SVyncRcTvfZWyn6lz1xMZEc7MWwfQsaV/3mDDkxTuIuLX\nZv6wk79/upnebRvz5oQBtGgY5u2SfILCXUT8kqPc8o/Pt/Dm8p0Mjz2bF8fE0SBUkXactoSI+J3C\nEgd3z0viy5T9TDgvhoeujCUkyIY6VkXhLiJ+JftYMbfNSmRjZnAPdayKwl1E/EbawWNMeGsNOfnF\nvDquPyN6tvJ2ST5L4S4ifmHFjhymzF5LaN06zJ08mLh2Tbxdkk9TuIuIz3s/MYP7P0ymQ4uzmDFh\nAO2aNfB2ST5P4S4iPstay/OLU/n30jTO79ScV8b1D5qrOlaXwl1EfFJRqYO/zt/IJxv2ckN8O564\npif1Qup4uyy/oXAXEZ+Tk1/M5FmJrNtzhPuu6MbvhnTEGA11PB0KdxHxKdsPHGPi22s4eLSYV8cF\n98W/qkPhLiI+47vUbO54dx1h9UKY9zuNiKkOhbuI+ITZq3bz6MIUOkc25M0JA2gbBPc5rUkKdxHx\nqjJHOU98toWZK3ZxabdIXhrbl4Zhiqbq0hYUEa85WlTKXe+t59vUbCae34EHftNd14jxEIW7iHhF\nxqECJs5cw86cn/jHNb24cVC0t0sKKAp3Eal1P+48xJR31lLmKGfWxIGc16mFt0sKOAp3EalV7ydm\n8LePkmnXtAFv3BKvuybVEIW7iNQKR7nl6S+3Mv27dC7o1IKXb+xH4wa6lEBNUbiLSI07VlTK1Dnr\n+WZbNjcPbs9DV8bqUgI1rMqta4yZYYw5aIzZdJL5xhjzkjEmzRiz0RjTz/Nlioi/2p37E9e+soLv\nt+fwxKiePJaga8TUBne28ExgxCnmXwF0dv5MBl6tflkiEghWpOWQ8PIPZOcXM2vSQG46t723Swoa\nVYa7tfY74NApmiQAs2yFVUATY4wuBiES5Gav3MX4GT/SsmEYC+44n/PO0YiY2uSJPve2QIbL80zn\ntH0nNjTGTKZi757oaI1pFQlEJWXlPPpJCu+t3sPQbpG8MCaOiHAdOK1tngj3yk4ns5U1tNZOB6YD\nxMfHV9pGRPxXTn4xf3hnHT/uOsSUi87hr5d31RmnXuKJcM8E2rk8jwL2emC5IuJHUvbmMXnWWnLy\ni3lxTBwJcW29XVJQ88Qh64XAzc5RM+cCedbaX3XJiEjg+mTDXka/uoJya5k/5TwFuw+ocs/dGDMH\nuBhoYYzJBB4B6gFYa18DPgdGAmlAAXBrTRUrIr7FUW557qttvLJsB/Htm/LKTf2IjAj3dlmCG+Fu\nrR1bxXwL3OGxikTEL+QVlvKnuRUnJo0dGM3fr+5BaF2NX/cVOkNVRE5b2sFj3D5rLRmHCnh8VE/G\na/y6z1G4i8hp+SplP3/+3wbC69XhvdvPZWCHZt4uSSqhcBcRt5SXW15YkspLS9PoHdWY18f3p3Vj\n3QrPVyncRaRKR4tKuXtuEl9vPcj1/aN4fFRPwuuFeLssOQWFu4ic0vYDx/jd7LXsOVTA4wk9uOnc\n9hijE5N8ncJdRE7q8+R93PP+BhqE1mXO5HMZEKP+dX+hcBeRXylzlPPsV9t4/dt0+kY34dVx/WnV\nWOPX/YnCXUR+ITe/mKlz1/NDWi43DormkatiCaur/nV/o3AXkZ9tyDjC799ZS85PJTxzXW9+G9+u\n6heJT1K4iwjWWuauyeCRBSm0jAjjgynn0SuqsbfLkmpQuIsEuaJSBw99vIn312ZyYecWvDSmL03P\nCvV2WVJNCneRILYnt4Dfv7uWlL1HmXppJ/44rIuuvx4gFO4iQerrLQe4e14SFnjzlniGdj/b2yWJ\nByncRYKMo9wybXEq//kmjR5tGvHquP5EN2/g7bLEwxTuIkEkJ7+YqXPWs2JHLjfEt+PvCT10GYEA\npXAXCRJrdh3izvfWcaSgVMMcg4DCXSTAWWv57/fpPP3lNto1rc9bfxhIbJtG3i5LapjCXSSA5RWU\n8pf3N7BkywFG9GjFM9f3plF4PW+XJbVA4S4SoJIz8/jDe2vZd6SIh6+M5dbzY3Q1xyCicBcJMNZa\nZq3czZOfbaFFw1D+N2Uw/aKberssqWUKd5EAcqyolPs+SOaz5H1c2i2S567vo7NNg5TCXSRAbMrK\n44731pF5uJD7rujG5As7UkdnmwYthbuIn3PthmneMJS5uqmGoHAX8Wt5haXcO38jX6bsVzeM/ILC\nXcRPrd9zmLvmrGd/XhF/G9mN2y5QN4z8P4W7iJ8pL7e8sTydZ77cxtmNwnl/ymD6ajSMnEDhLuJH\ncvKL+cv/NvBtajYjerTi6dG9adxAJyXJryncRfzED2k5/GleEnmFpTw+qic3DYrWSUlyUgp3ER9X\n6ihn2uJUXv12Bx1bnMWsiQPp3lrXhpFTU7iL+LCMQwXcNWc9SRlHGDOgHQ9fFUuDUP3ZStXquNPI\nGDPCGLPNGJNmjLmvkvnRxphvjDHrjTEbjTEjPV+qSHBZkJTFyBe/Z0d2Pv+5sS9Pje6tYBe3VflJ\nMcaEAC8Dw4FMYI0xZqG1drNLsweB/1lrXzXGxAKfAzE1UK9IwMsvLuORBSl8sC6T/u2b8sINcbRr\npjslyelxZzdgIJBmrU0HMMbMBRIA13C3wPFOwMbAXk8WKRIsNmQc4Y9z17PnUAFTh3Zm6qWdqBvi\n1hdskV9wJ9zbAhkuzzOBQSe0eRT4yhhzF3AWMMwj1YkECUe55bVvdzBtcSqREWHMuf1cBnVs7u2y\nxI+5E+6VjbWyJzwfC8y01j5njBkMzDbG9LTWlv9iQcZMBiYDREdHn0m9IgEn60ghf56XxOqdh/hN\n79b8Y1QvjV2XanMn3DMB15stRvHrbpdJwAgAa+1KY0w40AI46NrIWjsdmA4QHx9/4n8QIkFn4Ya9\nPPBRMuXllmev6811/aM0dl08wp1wXwN0NsZ0ALKAMcCNJ7TZAwwFZhpjugPhQLYnCxUJJEeLSnl0\nQQofrs+ib3QTXrghjvbNz/J2WRJAqgx3a22ZMeZOYBEQAsyw1qYYYx4DEq21C4G/AP81xtxNRZfN\nBGut9sxFKvHjzkPcPS+J/UeL+OPQztylg6ZSA9waNGut/ZyK4Y2u0x52ebwZON+zpYkElpKycqYt\nSeW1b3cQ3awB7+v2d1KDdEaESC1IPXCMP81NYvO+o4wZ0I6HrozlrDD9+UnN0adLpAaVl1tm/LCT\nZxZtIyKsLv+9OZ7hsWd7uywJAgp3kRqSebiAe97fwKr0QwzrfjZPje5Fi4Zh3i5LgoTCXcTDrLV8\nsC6Lvy9Modxanh7di9/Gt9MQR6lVCncRD8o+VswDHyXz1eYDDIxpxnO/7aPrwohXKNxFPOSL5H08\n8PEm8ovL+NvIbky6oCMhuqepeInCXaSajhSU8MjCFBYk7aVX28Y899s+dDk7wttlSZBTuItUw9Kt\nB7jvg2QO/VTC3cO68IdLzqGeTkgSH6BwFzkDeYWlPP7pZuavzaRbqwhmTBhAz7aNvV2WyM8U7iKn\nadm2g9z3QTLZ+cXccck5TB3ambC6Id4uS+QXFO4ibsorLOXJzzbzv8RMOkc25PXx/enTrom3yxKp\nlMJdxA1Ltx7gbx9u4uCxIqZcdA5/GtaZ8HraWxffpXAXOYUjBSU89ulmPlyXRZezG/L6+PO1ty5+\nQeEuchJfbtrPgx9v4nBBCXde0om7hnZS37r4DYW7yAmyjxXz6CcpfLZxH7GtGzHzVo2EEf+jcBdx\nstby0fosHvt0MwXFDu65rAu/u0jj1sU/KdxFqLiC4wMfbeLb1Gz6t2/K06N70ymyobfLEjljCncJ\nao5yy6yVu3h20TYAHrkqlpsHx+iaMOL3FO4StLbtP8Z9H25k/Z4jXNy1JU+M6klUU13BUQKDwl2C\nTlGpg/8sTeO1b3fQqH49XrghjoS4NrreugQUhbsElRU7cnjwo02k5/zEtf3a8uBvYml2Vqi3yxLx\nOIW7BIXDP5Xw5OdbmL82k+hmDZg1cSBDurT0dlkiNUbhLgHt+PDGJz7bwtHCUn5/8TlMvbQz9UN1\nMpIENoW7BKz07Hwe/HgTK3bk0je6Cf+4phfdWzfydlkitULhLgGnqNTBq8t28OqyHYTVq8MTo3py\n48Bo6mh4owQRhbsElO+3Z/PQx5vYlVvA1X3a8OCV3YmMCPd2WSK1TuEuAeHA0SIe/3Qzn27cR4cW\nZ/HOpEFc0LmFt8sS8RqFu/i1Mkc5b6/czbTFqZQ4yrl7WBd+d1FHXWtdgp7CXfxW4q5DPPjxJrbu\nP8ZFXVryWEIP2jc/y9tlifgEhbv4nZz8Yp76Yivz12bSpnE4r93Uj8t7tNIZpiIuFO7iN8oc5byz\najfPLU6lqNTBlIvOYerQTjQI1cdY5ERu/VUYY0YALwIhwBvW2qcqafNb4FHAAhustTd6sE4Jcj/u\nPMTDCyq6YC7s3IJHruqhS/KKnEKV4W6MCQFeBoYDmcAaY8xCa+1mlzadgfuB8621h40xkTVVsASX\n/XlF/POLLSxI2kubxuG8Oq4fI3qqC0akKu7suQ8E0qy16QDGmLlAArDZpc3twMvW2sMA1tqDni5U\ngktxmYMZy3fx76XbKSu3TL20E7+/uJMuGyDiJnfCvS2Q4fI8Exh0QpsuAMaYH6jounnUWvvliQsy\nxkwGJgNER0efSb0S4Ky1fL3lIE98tplduQUM6342D18ZS3RzXWdd5HS4E+6Vff+1lSynM3AxEAV8\nb4zpaa098osXWTsdmA4QHx9/4jIkyKUdPMZjn27hu9Rszml5lq7cKFIN7oR7JtDO5XkUsLeSNqus\ntaXATmPMNirCfo1HqpSAlldQygtfpzJr5W4ahIbw4G+6c8t5MboxtUg1uBPua4DOxpgOQBYwBjhx\nJMzHwFhgpjGmBRXdNOmeLFQCT5mjnDk/7uH5xankFZYyZmA0fxneheYNw7xdmojfqzLcrbVlxpg7\ngUVU9KfPsNamGGMeAxKttQud8y4zxmwGHMBfrbW5NVm4+LfvUrN5/NPNbD+Yz+COzXnoylhi2+hy\nvCKeYqz1Ttd3fHy8TUxM9Mq6xXu2HzjGk59vYdm2bNo3b8DfRnbnstizNbRRxE3GmLXW2viq2unU\nPqkVufnFvLBkO+/9uIcGoSE8MLI7N5/XnrC6GtooUhMU7lKjikodzPhhJ698s4PCUgc3DozmT8M6\nq19dpIYp3KVGlJdbFmzI4l+LUsk6Usiw7pHcd0U3OkVGeLs0kaCgcBePW7Ejh398voVNWUfp0aYR\nz17Xm/M66cYZIrVJ4S4es3X/UZ76YivLtmXTtkl9pt3Qh4Q+bXXvUhEvULhLte09Usjzi1P5YF0m\nEWF1uf+KbtxyXozuhiTiRQp3OWNHCkp4ddkO3lqxCyxMOr8Dd17aiSYNQr1dmkjQU7jLaSsscfDW\nip28tmwHx4rLuLZvFHcP70xUU13cS8RXKNzFbaWOcv6XmMGLS7Zz8FgxQ7tFcs/lXeneWmeWivga\nhbtUqbzc8snGvTy/OJXduQXEt2/Ky+P6MSCmmbdLE5GTULjLSR2/tvq/vtrG1v3H6NYqghkT4rmk\na6QuFyDi4xTuUqkf0nJ4dtE2kjKOENO8AS+OieOq3m00rFHETyjc5RfW7DrEc19tY1X6Ido0Duep\na3sxun+Urq0u4mcU7gJAUsYRnl+cynep2bRoGMYjV8UydmC0xqqL+CmFe5BLzsxj2pJUlm49SNMG\n9bj/im7cPDhGN6IW8XMK9yC1KSuPF5ZsZ8mWAzSuX4+/Xt6VW86LoWGYPhIigUB/yUFmU1YeL369\nncWbD9AovC5/Ht6FW8+PISK8nrdLExEPUrgHiY2ZR3jp6+0s2XLw51CfcH4MjRTqIgFJ4R7g1u05\nzL+/3s4327JpXL8efxnehVsU6iIBT+EeoFan5/LvpWksT8uhaYOKPvWbB7dX94tIkFC4BxBrLd9t\nz+E/S7ezZtdhWjQM428juzFuUHvO0oFSkaCiv/gAUF5u+Wrzfl7+ZgfJWXm0bhzO36/uwQ0D2mmc\nukiQUrj7sVJHOQuS9vLatztIO5hPTPMGPHVtL67tF0VoXZ1RKhLMFO5+qLDEwdw1e/jvd+nszSui\nW6sI/j22LyN7tSZE134RERTufuXwTyW8vXIXb6/YxeGCUgbENOXJa3pxcdeWukqjiPyCwt0PZBwq\n4M3lO5m3JoPCUgfDukfyu4vO0fXUReSkFO4+bFNWHtO/S+ez5H0Y4Oq4Nky56By6nB3h7dJExMcp\n3H2MtZZlqdn897t0VuzIpWFYXSZd0IFbz4+hdeP63i5PRPyEwt1HFJU6WJi0lzeWp5N6IJ9WjcK5\n/4pujB0UrbNJReS0Kdy9LCe/mHdX7WH2ql3k5JfQrVUEz13fh6v6tNFwRhE5Y26FuzFmBPAiEAK8\nYa196iTtrgPeBwZYaxM9VmUA2rb/GDOW7+SjpCxKysq5pGtLbruwI+ed01wjX0Sk2qoMd2NMCPAy\nMBzIBNYYYxZaazef0C4CmAqsrolCA0F5uWXp1oO8tWInP6TlEl6vDtf3j+LW82PoFKmDpCLiOe7s\nuQ8E0qy16QDGmLlAArD5hHaPA88A93i0wgBwtKiU+YmZzFq5i125BbRuHM5fL+/KjQOjaXpWqLfL\nE5EA5E64twUyXJ5nAoNcGxhj+gLtrLWfGmMU7k5pB/OZtXIXH6zN5KcSB/2im3DP5V25vEcr3XBa\nRGqUO+FeWQew/XmmMXWAacCEKhdkzGRgMkB0dLR7FfoZh7PrZdbKXXy/PYfQkDpc2ac1E86LoXdU\nE2+XJyJBwp1wzwTauTyPAva6PI8AegLLnAcCWwELjTFXn3hQ1Vo7HZgOEB8fbwkgufnFzEvM4N1V\ne8g6UkirRuHcc1kXxgyMpkXDMG+XJyJBxp1wXwN0NsZ0ALKAMcCNx2daa/OAFsefG2OWAfcEw2gZ\nay3r9hzhnVW7+Sx5HyVl5Qzu2JwHftOdy2LPpq66XkTES6oMd2ttmTHmTmARFUMhZ1hrU4wxjwGJ\n1tqFNV2kr8kvLmNBUhbvrtrD5n1HaRhWlzED2jH+3PZ01qUBRMQHuDXO3Vr7OfD5CdMePknbi6tf\nlm/avPco767ezcfrs/ipxEH31o148pqejIprqzsdiYhPUSJVoaCkjE827OW9HzPYkHGEsLp1uKpP\nG8YNiiauXROdcCQiPknhfhLJmXnMWbOHhUl7yS8uo3NkQx65KpZr+ralSQONTRcR36Zwd5FXWMrC\npCzmrskgZe9RwurW4Te9WzN2YDTx7ZtqL11E/EbQh3t5uWXVzlzeT8zk8+R9FJeVE9u6EY8l9CAh\nri2N6+uKjCLif4I23LOOFPLB2kzmr81kz6ECIsLrcn18FGMGRNOzbWNvlyciUi1BFe5FpQ4Wpexn\n/tpMlqflYC0M7ticPw/vwoierQivF+LtEkVEPCLgw91ay9rdh/lgXSafbtjHseIy2japz9RLO3Nd\n/yjaNWvg7RJFRDwuYMN9T24jQNG0AAAHQklEQVQBH67P5KP1WezOLaBBaAhX9GzN6P5tObdDc+rU\n0cFREQlcARXueQWlfJq8l4/WZZG4+zDGVHS73HVpZ67o2UonGolI0PD7tCsqdfDN1oN8nJTFN1uz\nKXGU0ymyIX+9vCuj+ralbRPdVFpEgo9fhruj3LIqPZcFSVl8sWk/x4rKaBkRxk3ntufafm3p0aaR\nxqSLSFDzu3BfuvUA936QTPaxYhqG1eWyHmdzTd+2nHdOC0LUjy4iAvhhuLdt0oB+0U1IiGvLpd0i\nNXxRRKQSfhfuXVtF8Pr4eG+XISLi03Q3CRGRAKRwFxEJQAp3EZEApHAXEQlACncRkQCkcBcRCUAK\ndxGRAKRwFxEJQMZa650VG5MN7D7Dl7cAcjxYjqf4al3gu7WprtOjuk5PINbV3lrbsqpGXgv36jDG\nJFprfe40VV+tC3y3NtV1elTX6QnmutQtIyISgBTuIiIByF/Dfbq3CzgJX60LfLc21XV6VNfpCdq6\n/LLPXURETs1f99xFROQUfDbcjTHXG2NSjDHlxpiTHlU2xowwxmwzxqQZY+5zmd7BGLPaGLPdGDPP\nGBPqobqaGWMWO5e72BjTtJI2lxhjklx+iowxo5zzZhpjdrrMi6utupztHC7rXugy3ZvbK84Ys9L5\nfm80xtzgMs+j2+tknxeX+WHO3z/NuT1iXObd75y+zRhzeXXqOIO6/myM2ezcPl8bY9q7zKv0Pa2l\nuiYYY7Jd1n+by7xbnO/7dmPMLbVc1zSXmlKNMUdc5tXk9pphjDlojNl0kvnGGPOSs+6Nxph+LvM8\nu72stT75A3QHugLLgPiTtAkBdgAdgVBgAxDrnPc/YIzz8WvA7z1U1zPAfc7H9wFPV9G+GXAIaOB8\nPhO4rga2l1t1Afknme617QV0ATo7H7cB9gFNPL29TvV5cWnzB+A15+MxwDzn41hn+zCgg3M5IbVY\n1yUun6HfH6/rVO9pLdU1AfhPJa9tBqQ7/23qfNy0tuo6of1dwIya3l7OZQ8B+gGbTjJ/JPAFYIBz\ngdU1tb18ds/dWrvFWrutimYDgTRrbbq1tgSYCyQYYwxwKTDf2e5tYJSHSktwLs/d5V4HfGGtLfDQ\n+k/mdOv6mbe3l7U21Vq73fl4L3AQqPIkjTNQ6eflFPXOB4Y6t08CMNdaW2yt3QmkOZdXK3VZa79x\n+QytAqI8tO5q1XUKlwOLrbWHrLWHgcXACC/VNRaY46F1n5K19jsqduZOJgGYZSusApoYY1pTA9vL\nZ8PdTW2BDJfnmc5pzYEj1tqyE6Z7wtnW2n0Azn8jq2g/hl9/sJ50fiWbZowJq+W6wo0xicaYVce7\nivCh7WWMGUjF3tgOl8me2l4n+7xU2sa5PfKo2D7uvLYm63I1iYq9v+Mqe09rs67RzvdnvjGm3Wm+\ntibrwtl91QFY6jK5praXO05Wu8e3l1fvoWqMWQK0qmTWA9baBe4sopJp9hTTq12Xu8twLqc10AtY\n5DL5fmA/FQE2HbgXeKwW64q21u41xnQElhpjkoGjlbTz1vaaDdxirS13Tj7j7VXZKiqZduLvWSOf\nqSq4vWxjzE1APHCRy+RfvafW2h2Vvb4G6voEmGOtLTbGTKHiW8+lbr62Jus6bgww31rrcJlWU9vL\nHbX2+fJquFtrh1VzEZlAO5fnUcBeKq7Z0MQYU9e593V8erXrMsYcMMa0ttbuc4bRwVMs6rfAR9ba\nUpdl73M+LDbGvAXcU5t1Obs9sNamG2OWAX2BD/Dy9jLGNAI+Ax50fl09vuwz3l6VONnnpbI2mcaY\nukBjKr5mu/PamqwLY8wwKv7DvMhaW3x8+kneU0+EVZV1WWtzXZ7+F3ja5bUXn/DaZR6oya26XIwB\n7nCdUIPbyx0nq93j28vfu2XWAJ1NxUiPUCreyIW24gjFN1T0dwPcArjzTcAdC53Lc2e5v+rrcwbc\n8X7uUUClR9Vroi5jTNPj3RrGmBbA+cBmb28v53v3ERV9ke+fMM+T26vSz8sp6r0OWOrcPguBMaZi\nNE0HoDPwYzVqOa26jDF9gdeBq621B12mV/qe1mJdrV2eXg1scT5eBFzmrK8pcBm//AZbo3U5a+tK\nxcHJlS7TanJ7uWMhcLNz1My5QJ5zB8bz26umjhpX9we4hor/zYqBA8Ai5/Q2wOcu7UYCqVT8z/uA\ny/SOVPzxpQHvA2Eeqqs58DWw3flvM+f0eOANl3YxQBZQ54TXLwWSqQipd4CGtVUXcJ5z3Ruc/07y\nhe0F3ASUAkkuP3E1sb0q+7xQ0c1ztfNxuPP3T3Nuj44ur33A+bptwBUe/rxXVdcS59/B8e2zsKr3\ntJbq+ieQ4lz/N0A3l9dOdG7HNODW2qzL+fxR4KkTXlfT22sOFaO9SqnIr0nAFGCKc74BXnbWnYzL\nSEBPby+doSoiEoD8vVtGREQqoXAXEQlACncRkQCkcBcRCUAKdxGRAKRwFxEJQAp3EZEApHAXEQlA\n/wf2u6GRITaWvgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8cb1978>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "e = 0.01\n",
    "s = np.linspace(-1, 1, 100)\n",
    "h = np.log(1+np.exp(s))\n",
    "\n",
    "plt.plot(s, h, label=r'$L=log(1+e^{s})$')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4VOWhx/HvS/aQkJAFkCQIKCAoixBAcQEFFBcuKCqu\niBtW1Fus7bWoVHux2gVaaisgWoqAChRckEK1KrRaRPYdgmFNICxJIAnZM3nvH4nckIAJZCZnZvL7\nPA+PzJwzc36D4fcc3nPmfY21FhER8S9NnA4gIiLup3IXEfFDKncRET+kchcR8UMqdxERP6RyFxHx\nQyp3ERE/pHIXEfFDKncRET8U6NSB4+LibNu2bZ06vIiIT1q3bl2mtTa+tv0cK/e2bduydu1apw4v\nIuKTjDH767KfhmVERPyQyl1ExA+p3EVE/JBjY+5nUlpaSnp6OkVFRU5HaTRCQ0NJTEwkKCjI6Sgi\n4kZeVe7p6elERkbStm1bjDFOx/F71lqysrJIT0+nXbt2TscRETeqdVjGGDPTGHPUGLP1LNuNMeZ1\nY0yqMWazMabn+YYpKioiNjZWxd5AjDHExsbqX0oifqguY+6zgCE/sP0moEPlrzHAtPoEUrE3LP15\ni/inWsvdWvtvIPsHdhkGzLYVVgHRxpgL3BVQRETOnTvG3BOAtCqP0yufy6i+ozFmDBVn97Rp08YN\nhxYR8W7WWorLyskvLqOgxEVBiYuosCBaRYV69LgNekHVWjsDmAGQnJyslbnP0549e/jVr35FTk4O\nCxcudDqO3/n8888BGDRokMNJxFsUlbrIzi/heEEJJwpKK34VlpBTWEpOYSm5hWXkFpWSV1TGyaJS\nThaXcbKojLzKQneVn153Twy4iOeGXOLRzO4o94NAUpXHiZXPiYe0b9+ev/zlL9xxxx1OR/FLr7zy\nCqBy93fWWrLzS8jIKeJwThFH84o5klvx38yTxRzLKyYrv5jskyXkl7jO+j7BAU1oFhZIs9AgIkMD\niQwNomWzUJqGBBIREkjTkACahgTSNDiQ8OCK33doEeHxz+eOcl8MPGWMmQf0BXKstTWGZHxJRkYG\nzzzzDKmpqeTm5pKQkMDy5cspLCxkyJAhfPnllwQEBNR4XUlJCYMGDeLLL78kMNCr7jIVaXS+L+/9\n2QUcyCrgQHYBadkFHDxRyMEThWTkFFFSVn7aa4yBmPBg4iNDiIsI4cLYcGKbhhAbEUxM02CahwcR\nHR5MdHgQ0WHBRIUFERrUxCtvTKi1gYwx7wMDgDhjTDrwEhAEYK2dDiwFbgZSgQLgIU+FbSgPPPAA\njz32GCNHjgRgy5YtAMycOZPbb7/9jMUOEBwczMCBA5k/fz733Xdfg+UVacxc5Zb9Wfl8d/QkqZW/\n9hw7yd7MfHKLyk7bt0VkCAnNw+iWGM2QS0O5ICqUVlFhtIoKpWWzikIPCvCPL+7XWu7W2ntq2W6B\nJ92WyGEul4sVK1Ywd+7cU8917doVgHfffZf33nvv1PPXX3892dkVNxLt3LmT2bNnM3z4cMaPH+/R\ncs/KyuKFF15gw4YNvPbaa4wfP95jxxLxJoUlLnYczmXbwRy2Hsxl5+FcUo7kUVT6/2fgF0SFclF8\nBMN6JNA2rint4sJpExNOYvNwQoPOfGLmjzR2UE1AQACDBg2ie/fuDB8+nFGjRnHVVVdRUlLCnj17\nqDoH/ZdffgnAtGnTWL58OSNGjABgzZo1Nd73mmuuIS8vr8bzkyZNOuex3djYWKZPn35OrxHxNdZa\n9mcVsHb/cTYcOM6GAydIOZJ36uJk8/AgurRuxn19L6RTq0g6tYzkohYRRISo1sCLy/2Xn2xj+6Fc\nt75nl9bNeGnopbXut2zZMv7zn/+wePFihgwZwpw5c+jTpw/R0dE19p09ezbLli1j0aJFp4ZrgoOD\nycvLIzIy8tR+X331lfs+iHjUm2++6XSERslay97MfP6zO4tVu7NYvS+bY3nFAESGBNI9KZon+l9E\n18QouiZEcUFUqFeOdXsLry13JxljuPrqq7n66qs5fvw4mzdvpn///jW+pv+3v/2Nd999l48//vi0\nibeKi4sJDT39Hta6nLk35A9qxWianEmnTp2cjtBo5BaV8vV3maxIOcpX32WSkVPxd+yCqFD6XRRL\n77Yx9G4bQ4cWETRpoiI/F15b7nU5w/aETz/9lOuuu47g4GCOHj3K119/zcyZM2nevDkul4uioiJC\nQ0NZsmQJU6dOZcmSJacVeVZWFnFxcTVmWazLmbunCnfz5s0sWLCAuLg4xo0b55Fj+JNPPvkEgKFD\nhzqcxD8dPFHIP7cd5rPtR/h2bzaucktkaCDXdIjjqYvjuOqiOC6MDddZeT15bbk7ZeHChYwdO5aI\niAhCQkKYOHEiV155JQA33HADX3/9NYMGDeLBBx8kJiaGq666CoCnn36aRx55hOXLl3PLLbec17Fr\n+3LSli1biIqKqvXbvR999BEfffQRADfddBPXXnstzZs3V7HX0eTJkwGVuztl5BTy980ZfLI5g01p\nJwDo0CKCMde25/pLWnB5UjSBfnKXirdQuVfz1ltvnXXbk08+yR/+8AcGDRpEVlbWGfd57733+PWv\nf31ex67ty0nr1q2jbdu2p5V7aWkpL730EgUFBZSXl/P6668zfPhwhg8ffmqfm2++meHDh5OdnU1M\nTMx5ZRM5V/nFZSzdksGi9ems2lNxV9llCc14bsgl3HhpS9rHe/6LPI2Zyv0c9OzZk+uuuw6Xy3XW\nLzENHz6cjh07uvW427dv5/XXXyclJYXIyEjmzZvHL3/5S1q2bMmMGTMoLCwkOjqavXv3nvH1S5cu\ndWsekR+yKe0E7367nyWbMygocdE2NpyfDO7I0O6taRfX1Ol4jYbK/Rw9/PDDZ90WHBzMqFGj3H7M\nLl26MH36dGbNmkXbtm0ZMGDAqW0bNmzgjTfeICQkxO3HFamr4jIXH288xJxv9rPlYA7hwQEM7daa\nu3on0rNNc42fO0Dl7kXO58tJw4YNY/To0SQlJXH99dczZMgPTb0v4l7Z+SXMXbWf2d/sJ/NkMR1b\nRvC/wy5l+OUJNAvV0o1OMk7dEpecnGzXrl172nM7duygc+fOjuRpzPTnfrq0tIoZrJOSkmrZs/E6\nklvEjH/v4b1vD1BY6uK6TvE8ek17+l2kldQ8zRizzlqbXNt+OnMXqUalfnZHc4v48/JU5q1Ow2Ut\nw3q05on+F9GhZWTtL5YGpXIXqWb+/PkApyaOE8gpKGXav3Yza+VeylyWO5MTeaL/xbSJDXc6mpyF\nyl2kmmnTKpYBVrlDqaucuav2M+Xz78gtKuW/urfmJ4M7cmGs7nrxdip3ETmjFSlHmbhkO7uP5XP1\nxXE8f3NnurRu5nQsqSOvK3drrS7INCDNMSPVHc4p4pefbGPZ1sO0i2vK26OSGdi5hf5e+hivKvfQ\n0FCysrKIjdUV94ZgrSUrK6vGJGfSOLnKLXO+2cekz3ZR6irnZzd24rFr2hMcqGkBfJFXlXtiYiLp\n6ekcO3bM6SiNRmhoKImJiU7HEIfty8znZws3sWbfcfp3jGfisMt0sdTHeVW5BwUF0a5dO6djSCN3\npknb/FV5ueWdb/bxm3/sJDigCb+/qzu3XZ6gfzn7Aa8qdxFvEBcX53SEBnE0r4hnF2ziq+8yua5T\nPK/d3o1WURqi8xcqd5FqZs2aBcDo0aMdzeFJy1OO8rO/bSKvqIxXhl/GfX3b6Gzdz6jcRarx53J3\nlVsmf5bC1BW7uaRVJO89dgUd9e1Sv6RyF2kksk4W8+N5G/k6NZN7+iTx0tBLCQ2qOXW1+AeVu0gj\nsPVgDmNmryUzv4TfjujGXb01f46/U7mL+LllWzJ4ZsFGYpuG8MET/bgsIcrpSNIAVO4ifspayxvL\nU5n02S56tonmzQeSiY/Uoi6NhcpdpBp/WJaw1FXO8x9s4W/r0hneozW/HtFN4+uNjMpdpJrwcN/+\nZmZBSRlPvrue5SnH+O+BHXhmUAfd5tgIqdxFqpk6dSoAY8eOdTjJucvOL+GhWWvYkn6CV2/ryr19\n2zgdSRyiGYFEqlmwYAELFixwOsY5O5pbxMg3v2FnRi7T7++lYm/kdOYu4gcOnijkvrdWcSyvmHce\n7sMV7WOdjiQOq9OZuzFmiDEmxRiTaoz5+Rm2RxljPjHGbDLGbDPGPOT+qCJyJvuz8rlr+jdk5Zcw\n59G+KnYB6lDuxpgA4A3gJqALcI8xpku13Z4EtltruwMDgMnGmGA3ZxWRatKyC7h7xioKSsp4/7Er\n6NmmudORxEvU5cy9D5Bqrd1jrS0B5gHDqu1jgUhTcUk+AsgGytyaVEROk378+2J38e6jV+jLSXKa\nuoy5JwBpVR6nA32r7fNnYDFwCIgERlpry92SUKSBrVixwukItcrIKeTet74lt6iU9x69QmubSg3u\nulvmRmAj0BroAfzZGFPjp80YM8YYs9YYs1arLYmcn6yTxdz39rcczy9hziN96ZqoM3apqS7lfhCo\nOstQYuVzVT0EfGArpAJ7gUuqv5G1doa1NtlamxwfH3++mUU8atKkSUyaNMnpGGd0sriMh2at4eDx\nQv4yujc9kqKdjiReqi7lvgboYIxpV3mR9G4qhmCqOgAMBDDGtAQ6AXvcGVSkoSxZsoQlS5Y4HaOG\n4jIXY2avZduhXKbe15M+7WKcjiRerNYxd2ttmTHmKeBTIACYaa3dZoz5UeX26cBEYJYxZgtggOes\ntZkezC3SqJSXW34yfxMrd2cx+c7uDOzc0ulI4uXq9CUma+1SYGm156ZX+f0h4Ab3RhOR7722bAd/\n35LBCzd3ZkSvRKfjiA/Q9AMiXm72N/t466u9PHjlhTx6TTun44iP0PQDItWEhYU5HeGUz7cf4eXF\n2xjUuSW/GHqpZneUOlO5i1SzbNkypyMAsP1QLv89bwOXJUTx+j09CGiiYpe607CMiBfKPFnMY7PX\n0iw0iLdHJRMerPMwOTf6iRGpZuLEiQBMmDDBkeMXl7l4Yu46svKL+dvj/WjRLNSRHOLbdOYuUs0X\nX3zBF1984cixrbX84qNtrNl3nN/d0V3fPpXzpnIX8SLvrT7A/LVpPHXdxQzt3trpOOLDVO4iXmL9\ngeO8vHgb/TvG88zgjk7HER+nchfxAsfyihk7dz2tokL54926M0bqTxdURaqJjW3YlYzKXOU8/f56\njheU8MHYfkSHa50bqT+Vu0g1ixYtatDj/eHzXazak83kO7tzaWtdQBX30LCMiIP+tesYbyzfzcjk\nJM0ZI26lchepZvz48YwfP97jx8nIKeSZ+Rvp1DKSl//rUo8fTxoXDcuIVPPNN994/BhlrnJ+/P5G\nikpdvHFfT8KCAzx+TGlcVO4iDvjTl6ms3pfNlJE9uLhFhNNxxA9pWEakga3em82fvvyO23smMPzy\nBKfjiJ9SuYs0oJyCUsbN20BSTDj/O+wyp+OIH9OwjEg1iYmeuWvFWsvzH27haF4xi57oR0SI/vqJ\n5+inS6SauXPneuR9F60/yN+3ZPDckEvonhTtkWOIfE/DMiINIC27gJcXb6NvuxjGXNve6TjSCKjc\nRaoZN24c48aNc9v7ucotzy7YBMDku7pr3hhpEBqWEalm48aNbn2/t77aw+p9FdMLJDYPd+t7i5yN\nztxFPGhHRi6TP0vhpstacXtP3fYoDUflLuIhJWXlPLtgE1FhQfzqtq4Yo+EYaTgalhHxkD8vT2V7\nRi4zHuhFTFNN4ysNS+UuUk3HjvVfBWlLeg5vLE/l9ssTuOHSVm5IJXJuVO4i1cyYMaNery8uc/Hs\n3zYSFxHMS0M126M4Q+Uu4mZ/+iKVXUdO8tfRvYkKD3I6jjRSuqAqUs2YMWMYM2bMeb1268Ecpv1r\nNyN6JnLdJS3cnEyk7nTmLlLNrl27zut1pa5yfrZwMzFNg5lwa2c3pxI5Nyp3ETeZvmI3OzJyefOB\nXlrkWhxXp2EZY8wQY0yKMSbVGPPzs+wzwBiz0RizzRjzL/fGFPFuu47k8fqX33Frtwu4UXfHiBeo\n9czdGBMAvAEMBtKBNcaYxdba7VX2iQamAkOstQeMMRpslEbDVW55btFmIkICtRaqeI26DMv0AVKt\ntXsAjDHzgGHA9ir73At8YK09AGCtPeruoCINpUePHue0/9xV+9lw4AR/GNmduIgQD6USOTd1KfcE\nIK3K43Sgb7V9OgJBxpgVQCTwR2vt7OpvZIwZA4wBaNOmzfnkFfG4KVOm1HnfgycK+e0/dnJtx3iG\n99DcMeI93HUrZCDQC7gFuBGYYIyp8TU/a+0Ma22ytTY5Pj7eTYcWcYa1lhc/3IIFXr3tMs0dI16l\nLmfuB4GkKo8TK5+rKh3IstbmA/nGmH8D3YHzu6dMxEH3338/UPuKTIs3HWJ5yjF+cWsXTeUrXqcu\nZ+5rgA7GmHbGmGDgbmBxtX0+Bq42xgQaY8KpGLbZ4d6oIg0jPT2d9PT0H9znREEJE5dsp3tiFA/2\na9swwUTOQa1n7tbaMmPMU8CnQAAw01q7zRjzo8rt0621O4wx/wA2A+XA29barZ4MLuKk15bu5HhB\nKbMf7quVlcQr1elLTNbapcDSas9Nr/b4d8Dv3BdNxDut2pPF/LVpPH5te7q0buZ0HJEz0twyIueg\nuMzF8x9uISkmjB8P6uB0HJGz0vQDItVceeWVZ902bcVu9hzL552H+xAerL8+4r300ylSzWuvvXbG\n53cfO8nU5bsZ2r01/TvqVl7xbhqWEakDay0TPtpKSFATzfgoPkHlLlLNiBEjGDFixGnPfbjhICt3\nZ/HckEtoERnqUDKRutOwjEg1WVlZpz0+nl/CK3/fweVtorm3j6bNEN+gM3eRWvzmHzvJKSzl1du6\n0kT3tIuPULmL/IC1+7KZtyaNR69uR+cLdE+7+A6Vu8hZlLrKeeHDrSRE65528T0acxepZuDAgQDM\n/HovKUfyeGtUsu5pF5+jn1iRaiZMmED68QIG//7fDO7SksFdWjodSeScaVhG5AxeXlyx0JiWzRNf\npTN3kWp6X3M9Ww/m8sdZ80mIDnM6jsh58clyHzBgQI3n7rrrLsaOHUtBQQE333xzje2jR49m9OjR\nZGZmcscdd9TY/sQTTzBy5EjS0tJ44IEHamx/9tlnGTp0KCkpKTz++OM1tr/44osMGjSIjRs3Mm7c\nuBrbX331Vfr168fKlSt5/vnna2yfMmUKPXr04PPPP+eVV16psf3NN9+kU6dOfPLJJ0yePLnG9jlz\n5pCUlMT8+fOZNm1aje0LFy4kLi6OWbNmMWvWrBrbly5dSnh4OFOnTmXBggU1tq9YsQKASZMmsWTJ\nktO2hYWFsWzZMgAmTpzIF198cdr22NhYFi1aBMD48eP55ptvTtuemJh4amGMcePGsXHjxtO2d+zY\nkRkzZgAwZswYdu06fQ2YHj16nFoa7/77768xF/uVV155akqBESNG1LiPfeDAgUyYMAGAG24cwoa1\nqzHAuxMe4T0Dt956Kz/96U8B/ezpZ889P3vffyZP0rCMSBX7svIpt5bQoAC0ap74MmOtdeTAycnJ\ndu3atY4cW+RMdh7O5ZbXv6Z08Uu0j2vaIGdXIufKGLPOWptc234+OSwj4m7l5ZYXPtxKVFgQ4TFa\nD1V8n8pdBFiwNo11+48z6c7u7Asb6nQckXpTuUujl3mymNeW7aRvuxhG9EzA9Pqp05FE6k0XVKXR\ne3XpDgpKyvjVbZdhdBVV/ITKXRq1lbsz+WD9QcZc256LW0QCFbc7numWRxFfonKXRqu4zMWEj7aS\nFBPG09drYjDxLxpzl0brzX/tYfexfGY91JvQoACn44i4lc7cpVHam5nPn5encku3CxjQqYXTcUTc\nTuUujY61lhc/2kJIQBNeurWL03FEPELDMtLofLzxEP9JzWLisEtp0azmYtd33XWXA6lE3EvlLo3K\niYISJi7ZTvekaO7te+EZ9xk7dmwDpxJxP5W7NCqvLt3BicJS5tzWlYCzLHZdUFAAQHi4piEQ36Vy\nl0Zj1Z4sFqxN5/H+7enS+uyLXX8/ba8mDhNfpguq0igUl7l4/sMtJMWEMW5gR6fjiHhcncrdGDPE\nGJNijEk1xvz8B/brbYwpM8bUXJFAxEFTl+9mz7F8XhnelbBg3dMu/q/WcjfGBABvADcBXYB7jDE1\n7h+r3O83wGfuDilSH98dyWPqilSG9WhN/47xTscRaRB1OXPvA6Raa/dYa0uAecCwM+z3NLAIOOrG\nfCL1Ul5ueW7RZiJCAvmF7mmXRqQuF1QTgLQqj9OBvlV3MMYkALcB1wG93ZZOpJ7mfruf9QdO8Pu7\nuhMbEVKn14wePdqzoUQagLvulpkCPGetLf+hKVONMWOAMQBt2rRx06FFzuzgiUJ+s2wn13SI47bL\nE+r8OpW7+IO6lPtBIKnK48TK56pKBuZVFnsccLMxpsxa+1HVnay1M4AZULGG6vmGFqmNtZYJH22l\n3MKrt3U9p3naMzMzAYiLi/NUPBGPq0u5rwE6GGPaUVHqdwP3Vt3BWtvu+98bY2YBS6oXu0hD+njj\nIb7ceZQJt3Yh6RzXRL3jjoqbvXSfu/iyWsvdWltmjHkK+BQIAGZaa7cZY35UuX26hzOKnJNjecW8\n/Mk2eraJZnS/tk7HEXFEncbcrbVLgaXVnjtjqVtrR9c/lsj5e2nxVgqKXfz2jm5nnWJAxN/pG6ri\nV5ZuyWDplsP8eFCHU8vmiTRGKnfxG8fzS/jFx1u5LKEZY65t73QcEUdp4jDxGxM+3kpOYSmzH+5L\nUMD5n7c88cQTbkwl4gyVu/iFv2/OYMnmDJ4d3PEHZ3ysi5EjR7oplYhzNCwjPu9YXjEvfrSFbolR\nPDHgonq/X1paGmlpabXvKOLFdOYuPu379VDzS1xMvrM7gfUYjvneAw88AOg+d/FtOnMXn/bhhoN8\nuu0Izw7uSIeWujtG5Hsqd/FZ6ccLeOnjbfRpG8Oj1+juGJGqVO7ik1zllp8s2IQFJt/VXV9WEqlG\n5S4+6S9f72H13mxeGnruc8eINAa6oCo+Z/uhXCZ9uosbL23JHb0S3f7+zz77rNvfU6ShqdzFpxSW\nuHj6/fVEhwfx2u3dzmkq37oaOnSo299TpKGp3MWn/O+S7ezJzGfuI32JaRrskWOkpKQA0KlTJ4+8\nv0hDULmLz/jH1gzeX32Ax/u356qLPbeQxuOPPw7oPnfxbbqgKj7h0IlCnltU8S3UZwfrjFqkNip3\n8XqlrnKeem89rnLL63dfTnCgfmxFaqNhGfF6kz5NYf2BE/zpnstpG9fU6TgiPkGnQOLVvthxhDf/\nvYf7r2jD0O6tnY4j4jN05i5eK/14AT9ZsIlLWzfjxVu6NNhxX3zxxQY7loinqNzFKxWVunhi7nrK\nyy1v3NuT0KCABjv2oEGDGuxYIp6ichevY63lFx9vZcvBHN4aldzg4+wbN24EoEePHg16XBF3UrmL\n13l/dRoL1qbz9PUXM7hLywY//rhx4wDd5y6+TRdUxatsOHCclxdv49qO8Ywb1NHpOCI+S+UuXiMj\np5Axc9bRMiqE1+/uoWl8RepB5S5eobDExZjZ6ygoLuMvD/YmOtwz88aINBYacxfHWWv52cJNbD2U\nw9ujkumo5fJE6k3lLo774xffsWRzBj+/6RIGdm74C6jVvfrqq05HEKk3lbs4auG6dKZ8/h0jeiby\n+LXesQ5qv379nI4gUm8acxfH/Cc1k58v2sxVF8fy2u1dPbLwxvlYuXIlK1eudDqGSL3ozF0ckXI4\njx/NWcdF8RFMu7+XV830+PzzzwO6z118m/f8jZJGIy27gFEzvyUsOICZD/WmWWiQ05FE/E6dyt0Y\nM8QYk2KMSTXG/PwM2+8zxmw2xmwxxqw0xnR3f1TxB5knixk1czWFJS5mP9KHhOgwpyOJ+KVay90Y\nEwC8AdwEdAHuMcZUn6JvL9DfWtsVmAjMcHdQ8X15RaU8OHM1GTmF/PWh3lzSqpnTkUT8Vl3O3PsA\nqdbaPdbaEmAeMKzqDtbaldba45UPVwGJ7o0pvq6gpIxHZq0l5XAe0+7vRa8LY5yOJOLX6nJBNQFI\nq/I4Hej7A/s/Aiw70wZjzBhgDECbNm3qGFF8XWGJi0dmrWXt/mxev+dyruvUwulIP2jKlClORxCp\nN7feLWOMuY6Kcr/6TNuttTOoHLJJTk627jy2eKeiUhdj5qxl1d4s/nBXD27t5v2rKWmqX/EHdSn3\ng0BSlceJlc+dxhjTDXgbuMlam+WeeOLLCktcPD53HV99l8nv7ujG8MsTnI5UJ59//jmgRTvEt9Wl\n3NcAHYwx7ago9buBe6vuYIxpA3wAPGCt3eX2lOJzThaX8eg7a/h2bza/GdGVO5OTan+Rl3jllVcA\nlbv4tlrL3VpbZox5CvgUCABmWmu3GWN+VLl9OvALIBaYWvktwzJrbbLnYos3yyks5aG/rmZTeg5T\nRvZgWA/fOGMX8Sd1GnO31i4FllZ7bnqV3z8KPOreaOKLjuYV8dBf17DrSB5v3NuTIZe1cjqSSKOk\n6QfEbfYcO8mDf11N1skS3n6wN/07xjsdSaTRUrmLW2xMO8HDs9ZggPcfu4LuSdFORxJp1FTuUm/L\ntmTwzIKNxEeGMPvhvrSLa+p0pHp58803nY4gUm8qdzlv1lreWJ7KpM920bNNNG8+kEx8ZIjTseqt\nU6dOTkcQqTeVu5yXwhIX4z/YzEcbDzG8R2t+PaIboUEBTsdyi08++QSAoUOHOpxE5Pyp3OWc7c/K\n5/E560g5ksfPbuzE2AEXec1CG+4wefJkQOUuvk3lLufkix1HGDd/I02M4a+jezPAy+eJEWmsVO5S\nJ8VlLn77jxT+8vVeLm3djOn39yIpJtzpWCJyFip3qdXuYyf57/c3sO1QLqOuvJDnb+7sN+PrIv5K\n5S5nVV5umfvtfl5bupPQoCa8NSqZwV1aOh1LROpA5S5nlJZdwP8s3Mw3e7K4tmM8vx3RjVZRoU7H\nahBz5sxxOoJIvanc5TRlrnLe+WY/v/8sBWMMv769KyN7J/nV3TC1SUrynRksRc5G5S6nbEo7wfMf\nbmHboVy/hZ8tAAAIO0lEQVQGdIrnV7d1bZQLWM+fPx+AkSNHOpxE5Pyp3IVjecX8/p8pzFuTRnxE\nCFPv68lNl7VqVGfrVU2bNg1QuYtvU7k3YkWlLt5ZuY8/fZlKUamLh/q1Y9zgDjQLDXI6mojUk8q9\nESpzlfPB+oNM+XwXh3KKGHhJC56/pTMXxUc4HU1E3ETl3oi4yi1LNh/i9S++Y/exfLonRvG7O7tz\n1cVxTkcTETdTuTcCpa5yPtxwkGkrdrM3M58OLSKYfn8vbry0ZaMdVxfxdyp3P5ZTUMp7qw/wzsp9\nHM4tqpw2oCc3dGlFkyYq9bNZuHCh0xFE6k3l7od2ZOTy7rf7+WD9QQpKXFx9cRyvjejKgI7xOlOv\ng7g4DVOJ71O5+4n84jKWbslg3po01u0/TkhgE27t1ppHrm5Hl9bNnI7nU2bNmgXA6NGjHc0hUh8q\ndx/mKres2pPFB+sPsmxrBgUlLtrFNeXFWzpzR69EosODnY7ok1Tu4g9U7j7GVW5Zuy+bv2/JYOmW\nw2SeLCYyJJBhPVozomcivS5srqEXEVG5+4L84jL+k5rJP7cf4YudR8nOLyEksAkDO7fg1m6tua5T\nC8KCNQWviPw/lbsXKi+3bM/IZeXuTP616xir92ZT6rJEhgZy/SUtuKFLK/p3iiciRP/7ROTM1A5e\noMxVzvaMXFbvzWbNvmy+3ZvNiYJSADq1jOThq9rRv1M8yRfGEBzYxOG0IuILVO4NzFrLoZwitqTn\nsDHtBBvTjrM5PYeCEhcAic3DGNy5Jf0ujqXfRXG0bNY45lD3JkuXLnU6gki9qdw9qKjUxe5jJ0k5\nnMfOw3nsyMhl26FcsvNLAAgKMHS5oBl39kqkV9sYerdtzgVRjW+KXW8THq61YcX3qdzrqbzcciSv\niH2ZBezNzGdfVj57jp3ku6MnScsuoNxW7Bcc2IQOLSIY3LkllyU049KEKLpc0ExrkXqhqVOnAjB2\n7FiHk4icP5V7LYpKXRzJLSIjp4jDOUUcPFHIwROFpB8vJP14AenZhZS4yk/tHxzYhLax4VzWOorh\nPRK4uEUEnS+IpG1sUwIDNF7uCxYsWACo3MW31ancjTFDgD8CAcDb1tpfV9tuKrffDBQAo621692c\n1S1c5ZbcwlKOF5RwvKCU7PwSsvOLycovIetkCcfyisk8WcyxvGKO5BaRW1RW4z1imgaTEB1Gp5aR\nDO7ckjax4bSJCaddXFNaR4Vp3hYRcVyt5W6MCQDeAAYD6cAaY8xia+32KrvdBHSo/NUXmFb5X7c7\nWVxGxolC8ktcFJSUUVDsIr+kjJPFZeQXl3GyqIy84jLyisrIKyolr6iM3KJScgpLOVFQysniMqw9\n83uHBQUQHxlCXEQw7eOb0u+iWFo0C6VFZAito8NoFRXKBVGhhAfrHzwi4t3q0lJ9gFRr7R4AY8w8\nYBhQtdyHAbOttRZYZYyJNsZcYK3NcHfg5TuP8vT7G8663RiICA6kaUggzcICaRYaRHxECBfHRxAd\nHkyzsCCiw4Jo3jSI6PBgYsKDiWkaTGxEsEpbRPxGXdosAUir8jidmmflZ9onAXB7ufe6sDl/uudy\nmoYEEBYUSNOQAJqGBBIRUlHo4UEBGhYRkUavQU9VjTFjgDEAbdq0Oa/3aB0dRuto3S4onrNixQqn\nI4jUW11u3zgIJFV5nFj53Lnug7V2hrU22VqbHB8ff65ZRUSkjupS7muADsaYdsaYYOBuYHG1fRYD\no0yFK4AcT4y3i4hI3dQ6LGOtLTPGPAV8SsWtkDOttduMMT+q3D4dWErFbZCpVNwK+ZDnIouISG3q\nNOZurV1KRYFXfW56ld9b4En3RhMRkfOlr0yKiPghlbuIiB9SuYuI+CGVu4iIH1K5i4j4IWPPNouW\npw9szDFgvyMHr584INPpEA1Mn9n/NbbPC777mS+01tb6LVDHyt1XGWPWWmuTnc7RkPSZ/V9j+7zg\n/59ZwzIiIn5I5S4i4odU7uduhtMBHKDP7P8a2+cFP//MGnMXEfFDOnMXEfFDKvd6MMY8a4yxxpg4\np7N4kjHmd8aYncaYzcaYD40x0U5n8hRjzBBjTIoxJtUY83On83iaMSbJGLPcGLPdGLPNGPNjpzM1\nFGNMgDFmgzFmidNZPEHlfp6MMUnADcABp7M0gH8Cl1lruwG7gPEO5/GIKovB3wR0Ae4xxnRxNpXH\nlQHPWmu7AFcATzaCz/y9HwM7nA7hKSr38/cH4H8Av79oYa39zFpbVvlwFRUrbfmjU4vBW2tLgO8X\ng/db1toMa+36yt/nUVF2Cc6m8jxjTCJwC/C201k8ReV+Howxw4CD1tpNTmdxwMPAMqdDeMjZFnpv\nFIwxbYHLgW+dTdIgplBxclbudBBPadAFsn2JMeZzoNUZNr0APE/FkIzf+KHPa639uHKfF6j4Z/y7\nDZlNPM8YEwEsAsZZa3OdzuNJxphbgaPW2nXGmAFO5/EUlftZWGsHnel5Y0xXoB2wyRgDFUMU640x\nfay1hxswolud7fN+zxgzGrgVGGj99/7ZOi307m+MMUFUFPu71toPnM7TAK4C/ssYczMQCjQzxsy1\n1t7vcC630n3u9WSM2QckW2t9cQKiOjHGDAF+D/S31h5zOo+nGGMCqbhgPJCKUl8D3Gut3eZoMA8y\nFWco7wDZ1tpxTudpaJVn7j+11t7qdBZ305i71MWfgUjgn8aYjcaY6bW9wBdVXjT+fjH4HcACfy72\nSlcBDwDXV/6/3Vh5Ris+TmfuIiJ+SGfuIiJ+SOUuIuKHVO4iIn5I5S4i4odU7iIifkjlLiLih1Tu\nIiJ+SOUuIuKH/g9ms2n2H+PepwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8f707b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "s = np.linspace(-5,5,100)\n",
    "h = 1/(1+np.exp(-s))\n",
    "\n",
    "plt.plot(s, h, label=r'$S(z)=\\frac{1}{1+e^{-z}}$')\n",
    "plt.legend()\n",
    "plt.plot([-5,5],[0.5,0.5],'k--')\n",
    "plt.plot([0,0],[0,1],'k--')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/../1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "根据之前计算得到的得分函数$s$（即$\\hat y$），若$s$很大，则$\\theta(s)\\approx 1$；若$s$很小，则$\\theta(s)\\approx 0$。通常来说，$\\theta(s)$与0.5比较，决定预测输出，从而进行分类。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- $\\theta(s)\\geq 0.5$：正类\n",
    "\n",
    "- $\\theta(s)<0.5$：负类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 代价函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "逻辑回归的代价函数（Cost Function）一般采用交叉熵（Cross-entropy Error）。常用的交叉熵形式为："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$J=-\\frac1m\\sum_{i=1}^m[y^{(i)}log\\hat y^{(i)}+(1-y^{(i)})log(1-\\hat y^{(i)})]$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上式中，$m$表示训练样本个数，$y^{(i)}$表示第$i$个样本真实标签，取值0或1，$\\hat y^{(i)}$表示第$i$个样本的预测输出，取值范围在(0,1)之间。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其实，对于交叉熵的解释非常简单。分为两种情况："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 当$y=1$时，$J=-\\frac1m\\sum_{i=1}^mlog\\hat y^{(i)}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此时，$J$的曲线类似于下图："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/../2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过上图可以很清楚看到，当$\\hat y=1$时，$J\\approx0$；当$\\hat y=0$时，$J\\approx\\infty$。由于该情况下，真实样本$y=1$，$J$的曲线正好与我们期望的一致，即$\\hat y$越接近1，损失函数$J$越小。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 当$y=0$时，$J=-\\sum_{(i=1)}^mlog(1-\\hat y^{(i)})$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此时，$J$的曲线类似于下图："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/../3.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "显然，当$\\hat y=1$时，$J\\approx\\infty$；当$\\hat y=0$时，$J\\approx0$。该情况下，真实样本$y=0$，$J$的曲线正好与我们期望的一致，即$\\hat y$越接近0，损失函数$J$越小。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "综上所述，代价函数$J$的数学表达式能够很好地反映$\\hat y$与$y$之间的偏差程度，偏差越大，“惩罚”越大！"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. 偏导数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "代价函数$J$的表达式为："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$J=-\\frac1m\\sum_{i=1}^m[y^{(i)}log\\hat y^{(i)}+(1-y^{(i)})log(1-\\hat y^{(i)})]$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "将$\\hat y=\\theta(s)$，$s=w_0x_0+w_1x_1+\\cdots+w_kx_k$代入，并写出矩阵形式："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$J=-\\frac1m[Ylog\\theta(XW)+(1-Y)log(1-\\theta(XW)]$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中，矩阵$W$是待求参数，计算$J$对$W$的偏导数："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\\frac{\\partial J}{\\partial W}=\\frac{\\partial J}{\\partial\\theta(XW)}\\cdot\\frac{\\partial\\theta(XW)}{\\partial W}=\\frac1m \\frac{\\theta(XW)-Y}{\\theta(XW)(1-\\theta(XW))}\\cdot X^T\\theta(XW)(1-\\theta(XW))=\\frac1m X^T(Y-\\theta(XW))=\\frac1m X^T(Y-\\hat Y)$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "推导过程并不复杂，只需记住$J$对$W$的偏导数为$\\frac1m X^T(Y-\\hat Y)$就好了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "得到偏导数之后，我们就能写出参数$W$的更新公式了："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$W=W-\\eta\\frac{\\partial J}{\\partial W}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中，$\\eta$是学习因子，即步进长度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. 逻辑回归实例"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据准备"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据集我已经存放在'../data/'目录下，该数据集包含了100个样本，正负样本各50，特征维度为2。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "data = pd.read_csv('./data.csv', header=None)\n",
    "# 样本输入，维度（100，2）\n",
    "X = data.iloc[:,:2].values\n",
    "m = X.shape[0]    # 样本个数\n",
    "# 样本输出，维度（100，）\n",
    "y = data.iloc[:,2].values.reshape((m,1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们在二维平面上绘出正负样本的分布情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEWCAYAAABliCz2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XuUHGW57/Hvk0nC5Ea4JAoSkxE3\nXkJMhmSScMtFgQhe8IIiGBcHj+ywuQ6s4BEMkKgb9vHIMqAckIjHoztxi3I7yiKA6EYDBHXCRRA4\nwlaCObAliSRAhsBcnvNHdc90d3pmqnq6uqu6fp+1es1UdXXVW1XJM9XPU+9b5u6IiEjjG1HvBoiI\nSG0o4IuIZIQCvohIRijgi4hkhAK+iEhGKOCLiGSEAr6knpl92cxurPayIdblZvYP1ViXSC2Y7sOX\nJDGz04HlwDuBV4DbgEvcfUc921WOmTlwiLs/W+a9+4DDgS7AgWeAnwKr3f2N4a5fpBK6wpfEMLPl\nwNeBLwITCQLmNOAXZjZ6gM+MrF0LIzvX3ScABxL8ETsFuNPMrL7NkqxSwJdEMLO9ga8A57n7Xe7e\n5e7PAScTBP3P5ZZbZWY3m9laM3sFOD03b23Buk4zs81mtt3MLjOz58zs2ILPr8393pJLy/wXM3ve\nzLaZ2YqC9cwzs41mtsPMXjSzawf6wzMYd9/l7vcBJwJHAB8eav1m9pvcxx8zs9fM7DNmtq+Z3WFm\nW83s5dzvU6K2R7JLAV+S4kigGbi1cKa7vwasB44rmP0x4GZgH2Bd4fJmNh24DlhKcGU9EThoiG0f\nDbwbOAa43Mzem5vfA1wITCII1McAZ0fcr8J9eR7oABYMtX53X5hbZpa7j3f3mwj+v36f4A/gVOB1\n4NpK2yPZo4AvSTEJ2Obu3WXeezH3ft5Gd7/d3Xvd/fWSZT8F/Nzd73f3N4HLCXLog/mKu7/u7o8B\njwGzANx9k7s/5O7duW8bNwCLou9akReA/SpZv7tvd/db3L3T3V8FrqhCeyRDkpz/lGzZBkwys5Fl\ngv6Buffz/jrIet5W+L67d5rZ9iG2/Z8Fv3cC4wHM7F3AN4E2YCzB/5dNQ6xrKAcBD1ayfjMbC6wG\njgf2zc2eYGZN7t4zzHZJBugKX5JiI/AG8MnCmWY2DjgB+GXB7MGu2F8E+vLaZjYG2L/CNl0PPE1w\np8zewJeBiguuZvZ2YA6wocL1LydIPc3PLZ9P+6gILKEo4EsiuPtOgqLtt83seDMbZWYtBLcybgH+\nNeSqbgY+amZH5gqgX6HygDiB4NbQ18zsPcBZlazEzMaa2SLg/wC/A+4Muf6/AQeXtOd1YIeZ7Qes\nrKQ9kl0K+JIY7v4/CK5yryIIhL8lSM8cE/bedXf/I3Ae8GOCq/1XgZcIvj1EdRHw2dw6vgvcFPHz\n15rZqwSB+2rgFuB4d+8Nuf5VwA9yd/GcnFvHGIL01kPAXVF3SLJNHa+koZnZeGAHQdrkL/Vuj0g9\n6QpfGo6ZfTSXRhlH8G3hceC5+rZKpP4U8KURfYzg9scXgEOAU1xfZUWU0hERyQpd4YuIZESiOl5N\nmjTJW1pa6t0MEZHU2LRp0zZ3nxxm2UQF/JaWFjo6OurdDBGR1DCzzWGXVUpHRCQjFPBFRDJCAV9E\nJCMSlcMvp6uriy1btrB79+56NyV1mpubmTJlCqNGjap3U0QkARIf8Lds2cKECRNoaWlBT4YLz93Z\nvn07W7Zs4R3veEe9myMiCZD4lM7u3bvZf//9FewjMjP2339/fTMSkT6JD/iAgn2FdNwa37p10NIC\nI0YEP9etG+oTkmWJT+mISHnr1sGyZdDZGUxv3hxMAyxdWr92SXKl4gq/3pqammhtbWXGjBl8+tOf\npjP/PyyCM844gyeffBKAK6+8sui9I488sirtlGxZsaI/2Od1dgbzRcpRwA9hzJgxPProozzxxBOM\nHj2a73znO5HXceONNzJ9+nRgz4D/4IMPVqWdki3PPx9tvkjDBfy4c5oLFizg2WefBeCb3/wmM2bM\nYMaMGVx99dUA7Nq1iw9/+MPMmjWLGTNmcNNNwUOMFi9eTEdHBxdffDGvv/46ra2tLM197x4/fjwA\nn/nMZ7jzzjv7tnX66adzyy230NPTwxe/+EXmzp3LzJkzueGGG6q7U5JKU6dGmy+Cu8f2InjoxOPA\no0DHUMvPmTPHSz355JN7zBvI2rXuY8e6Q/9r7Nhg/nCMGzfO3d27urr8xBNP9Ouuu847Ojp8xowZ\n/tprr/mrr77q06dP94cffthvvvlmP+OMM/o+u2PHDnd3X7Rokf/+978vWl/p+m+99VY/7bTT3N39\njTfe8ClTpnhnZ6ffcMMN/rWvfc3d3Xfv3u1z5szxP//5z6HaHuX4SbrE9e89C9audZ82zd0s+Jnm\nYxYmtuZftbjCf7+7t7p7W9wbiiunmb8ib2trY+rUqXzhC1/g/vvv5xOf+ATjxo1j/PjxfPKTn2TD\nhg28733v49577+VLX/oSGzZsYOLEiaG3c8IJJ/CrX/2KN954g/Xr17Nw4ULGjBnDPffcww9/+ENa\nW1uZP38+27dv55lnnhneTknqLV0Ka9bAtGlgFvxcs0YF26Hki92bNwd/JvPF7izc4dRQd+nEldPM\n5/AL+QAPjnnXu97Fpk2buPPOO7nkkktYsmQJl19+eajtNDc3s3jxYu6++25uuukmTj311L5tffvb\n3+aDH/zg8HZEGs7SpQrwUQ12YdjoxzLuK3wH7jGzTWa2rNwCZrbMzDrMrGPr1q3D2lgtc5oLFy7k\n9ttvp7Ozk127dnHbbbexYMECXnjhBcaOHcvnPvc5LrroIh5++OE9Pjtq1Ci6urrKrveUU07h+9//\nPhs2bOgL8B/84Ae5/vrr+z7zpz/9iV27dlV/p0QyIMvF7rgD/lHuPhs4ATjHzBaWLuDua9y9zd3b\nJk8ONYb/gK64AsaOLZ43dmwwv9pmz57N6aefzrx585g/fz5nnHEGhx12GI8//jjz5s2jtbWVK664\ngksvvXSPzy5btoyZM2f2FW0LLVmyhN/85jcce+yxjB49Gghu6Zw+fTqzZ89mxowZnHnmmXR3d1d/\np0QqkLbOX3FeGCb+WIRN9g/3BawCLhpsmeEWbd0bqxhTDSraSpzSWDiOq831OhYkoWhrZuPMbEL+\nd2AJ8ERc28tbuhSeew56e4OfjZ6TE6mnNHb+iqvYnYZjEWfR9q3AbbnxXEYCP3L3u2LcnojUWFrz\n4XEUu9NwLGIL+O7+Z2BWXOsXkfqbOjW4rbHc/KxJw7FouJ62ImmX+MJfgVreKFEvYc9HGo6FAr5I\ngqStU1Cjd/6Kcj7ScCzMB+hAVA9tbW3e0dFRNO+pp57ive99b51alH46funS0lI+LTBtWnATgtRW\nGs6HmW3ykCMZ6Ao/BDNj+fLlfdNXXXUVq1atqvp2NGyypKHwlyWNdj4aL+CXfmOpwjeYvfbai1tv\nvZVt27YNe12D0bDJohEwk6XRzkdjBfxVq+DCC/uDvHswPcyr8ZEjR7Js2TJWr169x3tbt27lpJNO\nYu7cucydO5cHHnigb/5xxx3H7NmzOfPMM5k2bVrfH4yPf/zjzJkzh0MPPZQ1a9YAaNhkAeIt/J19\nNowcGeSXR44MpqshTUXmvCQUYuty3ML20KrFa1g9bXt73dvbg+5t7e3lpys0btw437lzp0+bNs13\n7Njh3/jGN3zlypXu7n7qqaf6hg0b3N198+bN/p73vMfd3c855xy/8sor3d19/fr1DvjWrVvd3X37\n9u3u7t7Z2emHHnqob9u2rW87pdt1H96wyeppmz5x9BY/66ziHqD511lnDb+tjd7TNo7zUc3jRoSe\ntnUP8oWvYQ+tUBjk869hBnv3/sB72WWX+Ve/+tWigD958mSfNWtW3+ttb3ubv/LKKz5r1qyi4Lvv\nvvv2BfyVK1f6zJkzfebMmb733nv7xo0bi7ZTut3XX3/dp0yZ4rt37/bbb7/dP/vZz7q7+0knneSH\nHHJI37ZbWlr87rvvLlqHAr64uzc1lQ/4TU3DW++0aeXXO21aNVodjyS0uZptiBLwG2p4ZMxg9Wq4\n5pr+eatXB/Or4IILLmD27Nl8/vOf75vX29vLxo0bGTNmTNGywXnY03333ce9997Lxo0bGTt2LIsX\nL2b37t2DblfDJstw9fREmx9WGouaSWhzvdrQWDl8z+XsCxXm9Idpv/324+STT+Z73/te37wlS5Zw\n7bXX9k3nx80/+uij+clPfgLAPffcw8svvwzAzp072XfffRk7dixPP/00Dz30UN9nNWyyxKWpKdr8\nsNJY1ExCm+vVhsYJ+Plgf8010N4ejJ7W3h5MVzHoL1++vOhunW9961t0dHQwc+ZMpk+f3veA85Ur\nV3LPPfcwe/Zs1q9fz4EHHsiECRM4/vjj6e7uZubMmVx22WUcfvjhfevSsMkSl2Vln0Yx8PywklTU\nTEIhNqy6tSFs7qcWr2Hn8FeuLM7Z53P6uXx7Le3evdu7urrc3f3BBx/0WbNm1bwN7srhS2DtWveR\nI4vzxSNHVq8AWe+iZhIKsVFVqw1EyOE3Xk9b9+Kcfel0jTzzzDOcfPLJ9Pb2Mnr0aK677jrmzp1b\n83aop61AOnqMFora3rTtXzVF6WnbWEVb2DO41yHYAxxyyCE88sgjddm2SKkkFCqjiNretO1fvaQi\nh5+kbyFpouMmeUkoVEYRtb1p2796SXzAb25uZvv27QpeEbk727dvp7m5ud5Nqbq4eihGWW9SepeG\n7T0btUhY72Mctb1JKMSmQeJTOlOmTGHLli1s3bq13k1JnebmZqZMmVLvZlRVfrja/KPk8sPVwvCG\noY2y3rjaENXZZ8P11/dP9/T0T193XfGy+XatWBGkOaZODYJhufYm4RhHaW8ly2dV4ou2IoXiKs5F\nWW9SCoQjR5bvONXUBMO5OzcJx1jCi1K0VcCXVBkxonyXCrOg60Ut1htXG6Ia7H6E4fy3TsIxlvA0\nHr40rKjFubA54yjrTUqBMCm9Z+M4xpVIYw2m5sLesF+LV7mOVyKFonSwScKycUrCCJhJOW5JaUc9\n0EijZYqUCttDMeqIhFF6Piahp6Z7ENzzI2E2NQ0/2Ocl4RhHEaUdSRgts5qiBHzl8KVhKWccv6Qc\n4zTWYKpFOXwRkpNrb2RJOcZprMHUgwK+NCx1xqlcXB2k4iqWRmlHpv9dhM391OKlHL5UW1Jy7WkS\n18iTcRdL01iDqQaUwxeRSqnjVboohy8iFYtr5EmNaFl/CvgiUiSuomaWi6VJoYAvkhOloBh2lMq4\n2xGHuIqamS6WJkXYZH8tXiraSr1EKSjG1cM1ajviFFdRs5GKpUmBirYi0UQpKMY1SmXUdoiAirYi\nkUUpKJYL9oPNj6sdIlEp4IsQraAY1yiVUdshEpUCvghB4XD06OJ5o0eXLyjmn9IUdn6UIqwKmxIn\nBXyRnNJy1kDlraOOCvL4hUaODOaXyj/Wb/PmYH35x/oNFPSXLoU1a4KcvVnwc80aPapPqkNFWxHi\ne8ShirASNxVtRSKKUiyNa1mRuMUe8M2sycweMbM74t6WSKmwHaTiGl5Xj/WTJKnFFX478FQNtiNS\n5Oyz4frr+2+X7OkJpssF/biG1/3Qh8q3baD5UUSpD0StJUiDCttDq5IXMAX4JfAB4I6hlldPW6mm\n/KP/Sl9NTeWXj2N43Tgfp5flx/pJP5LS09bMbgb+BZgAXOTuHymzzDJgGcDUqVPnbC5X4RKpgNnA\n79XqXoU4H6eX5cf6Sb9EFG3N7CPAS+6+abDl3H2Nu7e5e9vkyZPjao5kUJwdpMKKM4efpFqCpEOc\nOfyjgBPN7Dngx8AHzGxtjNuTBElCgTDODlJhRenQVcm69Vg/iSRs7mc4L2AxyuFnRlJGfHQPRrDM\n5/KbmgYe0TKuNq9d6z5qVPF6R43SY/2kekhKDj/PzBYzQA6/kDpeNYY0djbSY/0kraLk8NXTVqou\njQXCuNqcxmMh6ZKIoq1kVxoLhHqsn2SBAr5UXZIKhGELsXqsn2RC2GR/LV4q2jaOJBQIoxZi9Vg/\nSSOSVrQNSzl8qSYVTCULlMMXQSNVipRSwJeGpYKpSDEFfGlYV1wBo0YVzxs1SgVTyS4FfGlopQOo\nDTagmkijU8CXhrViBbz5ZvG8N98M5otkkQK+NCwVbUWKKeBLw4patE3CCJ8icVLAl4YVpZerHgEo\nWaCALw1r6VJYsyboaGUW/FyzJphfasUK6OwsntfZqXy/NBb1tBVBo1pKeqmnrUhE6qQlWaCAn3Eq\nVAaijmqp4yZpNLLeDZD6yRcq87nrfKESyue5G1l+f1esCG7bnDo1CPbljoOOm6SVcvgZptEkK6Pj\nJkmiHL6Eoo5JldFxk7QaMOCb2fvM7CEz+6uZrTGzfQve+11tmidxUqGyMjpuklaDXeFfD6wC3gf8\nCbjfzN6Ze2/UQB+S9NDj9yqj4yZpNVjAH+/ud7n7Dne/CjgXuMvMDgeSk/iXikXpmCT9dNwkrQYs\n2prZY8BCd99ZMG8mcAuwn7vvX+3GqGgrIhJNtYq2XwfeWzjD3f8AHAPcWnnzRESkHga8D9/dfzTA\n/OeBf4ytRSIiEgvdltmAovQCVY9RkexQT9sGE6UXqHqMimSLeto2mCi9QNVjVCT9qtrT1szeZWa/\nNLMnctMzzezS4TZS4hGlF6h6jIpkS5gc/neBS4Au6LtT55Q4GyWVi9ILVD1GRbIlTMAf6+6lQyl0\nx9EYGb4ovUDVY1QkW8IE/G25IRUcwMw+BbwYa6ukYlF6garHqEi2DFm0NbODgTXAkcDLwF+Ape5e\nptw3PCraiohEE6VoO+htmWY2Amhz92PNbBwwwt1frUYjRUSktgZN6bh7L8Ggabj7LgV7EZH0CpPD\n/4WZXWRmbzez/fKv2FsmIiJVFaan7X/N/TynYJ4DB1e/OSIiEpchA767v6MWDRERkXgNGfDN7LRy\n8939h9VvjoiIxCVMDn9uwWsBwWMPTxzqQ2bWbGa/M7PHzOyPZvaVYbVU6k4ja4qkW5iUznmF02Y2\nEfjXEOt+A/iAu79mZqMInom73t0fqqypUk8aWVMk/SoZD78TOGSohTzwWm5yVO6VnKE5JZIVK/qD\nfV5nZzBfRNIhTA7/5/QH6hHAdOCnYVZuZk3AJuAfgP/p7r8ts8wyYBnAVI3alVgaWVMk/cLclnlV\nwe/dwGZ33xJm5e7eA7Sa2T7AbWY2w92fKFlmDcHQDbS1tekbQEJNnVp+7Hz9jRZJjzApnQ+5+69z\nrwfcfYuZfT3KRtx9B3AfcHwljZT608iaIukXJuAfV2beCUN9yMwm567sMbMxwLHA09GaJ0mhkTVF\n0m/AlI6ZnQWcDRxsZn8oeGsC8ECIdR8I/CCXxx8B/MTd7xhOY6W+li5VgBdJs8Fy+D8C1gP/Alxc\nMP9Vd//7UCvOPRnrsOE1T0REqmXAgO/uO4GdwKkAZvYWoBkYb2bj3V33Z4iIpEiYh5h/1MyeIXjw\nya+B5wiu/EVEJEXCFG3/GTgc+FNuILVjCJfDFxGRBAkT8LvcfTswwsxGuPu/A60xt0tERKosTMer\nHWY2HtgArDOzlwg6YImISIqEucL/GMH4ORcAdwH/AXw0zkaJiEj1hRktc5eZTQMOcfcfmNlYoCn+\npomISDWFuUvnH4GbgRtysw4Cbo+zUSIiUn1hUjrnAEcBrwC4+zPAW+JslIiIVF+YgP+Gu7+ZnzCz\nkWhcexGR1AkT8H9tZl8GxpjZcQRj4f883maJiEi1hQn4FwNbgceBM4E7gUvjbJSIiFTfYKNlTnX3\n5929F/hu7iUiIik12BV+3504ZnZLDdoiIiIxGizgW8HvB8fdEBERiddgAd8H+F1ERFJosJ62s8zs\nFYIr/TG538lNu7vvHXvrRESkagZ7AIqGTxARaSBhbssUEZEGoIAvIpIRCvgiIhmhgC8ikhEK+CIi\nGaGALyKSEQr4IiIZoYAvIpIRCvgiIhmhgC8ikhEK+CIiGaGALyKSEQr4IiIZoYAvIpIRCvgiIhmh\ngC8ikhEK+CIiGaGALyKSEQr4IiIZoYAvIpIRCvgiIhkRW8A3s7eb2b+b2VNm9kcza49rWyIiMrSR\nMa67G1ju7g+b2QRgk5n9wt2fjHGbIiIygNiu8N39RXd/OPf7q8BTwEFxbU9ERAZXkxy+mbUAhwG/\nLfPeMjPrMLOOrVu31qI56eQ++HRW2iAiFYs94JvZeOAW4AJ3f6X0fXdf4+5t7t42efLkuJuTTqtW\nwYUX9gdY92B61apstUFEhiXWgG9mowiC/Tp3vzXObTUsd9ixA665pj/gXnhhML1jR22uspPQBhEZ\nNvOY/rOamQE/AP7u7heE+UxbW5t3dHTE0p5UKwywee3tsHo1mGWnDSKyBzPb5O5toZaNMeAfDWwA\nHgd6c7O/7O53DvQZBfxBuMOIgi9kvb21D7RJaIOIFIkS8OO8S+d+dzd3n+nurbnXgMFeBpG/ui5U\nmE/PShtEZFjU0zbpClMp7e3BVXV7e3E+PQttEJFhi7PjlVSDGeyzT3G+fPXq4L199qlNSiUJbRCR\nYYsth18J5fAH4V4cWEuns9IGESmSiBy+VFlpYK1HoE1CG0SkYgr4El5Setr29g4+XQtJORYiESjg\nSzhJ6Wm7eDHMmdMf5Ht7g+nFi2vXhqQcC5GIFPBlaEnpadvbCzt3wqOP9gf9OXOC6Z07a3Oln5Rj\nIVIBFW0lnKT0tC0M8nmtrbBpU3GnsDgl5ViIkJCetpVQwE+4pPS07e2Fpqb+6Z6e2gX7vKQcC8k8\n3aWTFlEKf3EVKnt6Bp/Oc4cLSoZEuuCC2qcw8lf4hQpz+rWgXseSUgr49RKl8BdXobKlBQ44oD/I\n9/QE0y0txcu5wxFHwLe+BeefH2z//POD6SOOqG0OP5/OaW0N2tvaWpzTj5t6HUuKKeDXQ5TCX1yF\nyp4e2LULtm3rD/oHHBBM79o18JV+PY0YARMnFufsN20KpidOrE1aZ6Bex+3t6nUsyefuiXnNmTPH\nM6O317293T0I78GrvT2YX6qnx721tXjZ1tZg/nB0d7tPmlS83kmTgvnl2nv++cXLnn9++fbGrXS/\nh3scKlG63/U4DiLuDnR4yBirom09RSn8xVWo7OmBkQVDKnV3F2+n0vaKSE2oaFtPYQuxUQp/UQuV\nYQu8+TROocKcfml7oxRtoxaZk1DAjkI9bSWFFPCrKWwhNkrhL2qhMmyBt6cH9t47yNlPmhRc2U+a\nFEzvvXdx0HcPcuTlirYTJ+4Z7KIWmZNQwI5CPW0lpRTwqyVKITZK4S9KoTJKgXfEiP7UzSmnBNOn\nnBJMNzXtud7u7uD3deuC6XXrgunu7uL1Ri0yJ6GAHYV62kqahU321+KV+qJtlEJsfvnBpguFLVRG\nKfD29rqfe27xsueeW74d3d3u++9fvOz++5cv8EYtMiehgB1F1PMsEiNUtB2Cxziue5QiaBS9vXte\ndQ9UtO3uhlGj+qe7uorbVGl7u7pg9Oj+6TffLN5OpeuF+ArYcZ1rFbAlIVS0HUyc+ddFi8oXQRct\nGt56o+StFy2CCROK502YUL4NUdq7YAGMGVM8b8yYYH6pyy8vv97LL99zWYivgB3XuY7SXpEkCftV\noBav2FM6hV/F81/BS6crVXhPe/5e9tLpShSmMPKpi9LpvK4u9+bm4L3m5vLTlbT3zTfdm5qC95qa\nyk9XehyinJMoxyKucx3nvyGRChAhpVP3IF/4qkkOP87862WXle/IdNllw1tvlLz1ggX9QT7/am4O\n5g+nvVOn9gf5/KupKZhfauHC8utduLD8/q1cWXwO8udo5co9l120qHjf88dm0aI9l43rXEdpr0jM\nogT87Obw48q/xpnDD5u3TmsOP2yuPUo9I65zHaW9IjFSDn8w7sHtj4Xa2wfvIDXYdKHeXmgrOe5t\nbQPfLhh23VHy1r29MHdu8by5cwdeNmx7e3th3rziefPmlV/WHZYvL563fPngxy7K83LDLuse3wif\ner6vpFHYrwK1eNUkh3/QQcFX+/POC6bPOy+YPuigPb/qR/nqHiW/HGXdUdYbddkDDii/7AEHVL7e\nuHPcYY9bb6/7/PnBdvNj/uTHApo/X7l2aRhESOlk6wrfvb8H6YYNwfSGDcF0T0/xlZ97tA42UTpI\nRVl3lPVGWdasP8WyYEEwnb/jpqmp8s5fcY4mGfWciEixsH8ZavGqSdE2asekqEW/sB2koq47ygiR\nUdqQ/4aTf+W/+Qy3DXGNJhnluCVphE+RmKCi7RCidtyJq8Ab57rT1IaoorQ5jfsnEoGKtoOJUgDN\npwwKVauDTZzrjtKGJDy2MIooxy0JxzjfjsGmRWol7FeBWrxiT+kkpfiYhM47aSxqRjluSTjG7rpn\nX2JHhJTOADdnN6iBio9z5oQvPsLwi49xrruRRTluSTjGXlBkhmD7hcNiu+7dl9rKbg4/SseduDrY\nxLnusNu/4IJgXPu888+Hq69OdiCKctyScIzzQT6v8I+QyDBFyeFnM+BLPxU146djLDHKZtFWhbHo\nklLUbGQ6xpIgjRHw9ci56ApTDUM9ZlEqo2MsCZP+oq0KY5VJQlGz0ekYS8I0Rg5fhbHK1buomQU6\nxhKjbBZtk1IY039uEamhRBRtzex/mdlLZvZEXNvok5TCmGoJIpJgcRZt/zdwfIzrDySlMFZYS9BI\njiKSQLEVbd39N2bWEtf6+ySlMFa43Wuu6a8nqJYgIgkRaw4/F/DvcPcZgyyzDFgGMHXq1DmbN2+u\nbGNJyZ0npZYgIpmQiBx+WO6+xt3b3L1t8uTJla8oCY+cS0otQUSkjLoH/IaRlFqCiMgA0t/xKimS\nUksQERlAbDl8M/s3YDEwCfgbsNLdvzfYZxpi8LSk1BJEJBOi5PDjvEvn1LjWnWhJqCWIiJShHL6I\nSEYo4IuIZIQCvohIRijgi4hkhAK+iEhGKOCLiGREosbDN7OtQIWD6cRuErCt3o2IkfYv3Rp5/xp5\n32D4+zfN3UONS5OogJ9kZtYRtnNDGmn/0q2R96+R9w1qu39K6YiIZIQCvohIRijgh7em3g2ImfYv\n3Rp5/xp536CG+6ccvohIRujfaNa6AAAFaklEQVQKX0QkIxTwRUQyQgG/DDNrMrNHzOyOMu+dbmZb\nzezR3OuMerSxUmb2nJk9nmv7Hg8fsMC3zOxZM/uDmc2uRzsrFWL/FpvZzoLzd3k92lkJM9vHzG42\ns6fN7CkzO6Lk/bSfu6H2L83n7t0F7X7UzF4xswtKlon9/OmJV+W1A08Bew/w/k3ufm4N21Nt73f3\ngTp6nAAcknvNB67P/UyTwfYPYIO7f6Rmramea4C73P1TZjYaGFvyftrP3VD7Byk9d+7+f4FWCC4o\ngf8H3FayWOznT1f4JcxsCvBh4MZ6t6VOPgb80AMPAfuY2YH1blTWmdnewELgewDu/qa77yhZLLXn\nLuT+NYpjgP9w99JRBWI/fwr4e7oa+G9A7yDLnJT7ynWzmb29Ru2qFgfuMbNNZraszPsHAX8tmN6S\nm5cWQ+0fwBFm9piZrTezQ2vZuGE4GNgKfD+XbrzRzMaVLJPmcxdm/yCd567UKcC/lZkf+/lTwC9g\nZh8BXnL3TYMs9nOgxd1nAvcCP6hJ46rnKHefTfD18RwzW1jyfrlnMqbp3t2h9u9hgrFHZgHfBm6v\ndQMrNBKYDVzv7ocBu4CLS5ZJ87kLs39pPXd9cqmqE4Gflnu7zLyqnj8F/GJHASea2XPAj4EPmNna\nwgXcfbu7v5Gb/C4wp7ZNHB53fyH38yWCHOK8kkW2AIXfWqYAL9SmdcM31P65+yvu/lru9zuBUWY2\nqeYNjW4LsMXdf5ubvpkgQJYuk9ZzN+T+pfjcFToBeNjd/1bmvdjPnwJ+AXe/xN2nuHsLwdeuX7n7\n5wqXKcmpnUhQ3E0FMxtnZhPyvwNLgCdKFvsZcFrujoHDgZ3u/mKNm1qRMPtnZgeYBU+WN7N5BP8H\ntte6rVG5+38CfzWzd+dmHQM8WbJYas9dmP1L67krcSrl0zlQg/Onu3RCMLOvAh3u/jPgfDM7EegG\n/g6cXs+2RfRW4Lbc/5mRwI/c/S4z+ycAd/8OcCfwIeBZoBP4fJ3aWokw+/cp4Cwz6wZeB07x9HQ3\nPw9Yl0sL/Bn4fAOdOxh6/9J87jCzscBxwJkF82p6/jS0gohIRiilIyKSEQr4IiIZoYAvIpIRCvgi\nIhmhgC8ikhEK+NIwzKynZETClgrWsY+ZnV391vWt/z1mttHM3jCzi+Lajkg5ui1TGoaZvebu44e5\njhbgDnefEfFzTe7eE2K5twDTgI8DL7v7VZW0U6QSusKXhmbBsw2+YWa/zw14d2Zu/ngz+6WZPWzB\n+Pkfy33kvwPvzH1D+IYFY7DfUbC+a83s9Nzvz5nZ5WZ2P/BpM3unmd2VG7htg5m9p7Q97v6Su/8e\n6Ip950VKqKetNJIxZvZo7ve/uPsngC8QdFGfa2Z7AQ+Y2T0EoxJ+wt1fyY3H8pCZ/YxgwK4Z7p4f\nu3zxENvc7e5H55b9JfBP7v6Mmc0HrgM+UO2dFKmUAr40ktfzgbrAEmCmmX0qNz2R4AETW4Arc6Np\n9hIMQ/vWCrZ5EwTfGIAjgZ/mhnYA2KuC9YnERgFfGp0B57n73UUzg7TMZGCOu3flRkhtLvP5bopT\nn6XL7Mr9HAHsKPMHRyQxlMOXRnc3wYBbowDM7F25kTQnEjz7oMvM3k9QSAV4FZhQ8PnNwHQz28vM\nJhKM4rgHd38F+IuZfTq3HTOzWfHskkhldIUvje5GoAV4ODe07laCO2TWAT+34EHnjwJPQ/C8AzN7\nwMyeANa7+xfN7CfAH4BngEcG2dZS4HozuxQYRfBMhccKFzCzA4AOgucl91rwIOvpuT8YIrHSbZki\nIhmhlI6ISEYo4IuIZIQCvohIRijgi4hkhAK+iEhGKOCLiGSEAr6ISEb8f7vahkhdNuLQAAAAAElF\nTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xbb0a908>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.scatter(X[:50, 0], X[:50, 1], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 0], X[50:, 1], color='red', marker='x', label='Negative')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.title('Original Data')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 特征归一化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先分别对两个特征进行归一化处理，即："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$X=\\frac{X-\\mu}{\\sigma^2}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中，$\\mu$是特征均值，$\\sigma^2$是特征方差。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEWCAYAAABBvWFzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XucVXW9//HXm0FExAsC53hBwDqU\nEuIoI2rm5Zc3rH5qZQVS6e/owSwTTTvpjxSOHTuen/3SOpVHzGOWnLKLmZmoeelEpsVQWF5SyUvy\nw5NIogmizPD5/bHWhj3Dnpm9ZvaavfbM+/l4rMfMun/2Gt0f1vezvt+liMDMzKyvhtQ7ADMzGxic\nUMzMrCacUMzMrCacUMzMrCacUMzMrCacUMzMrCacUMxqQNICSTemv4+X9Kqkphqf4zBJj9fymN2c\n6xlJR/fHuWzgcEKxhpB+wf1Z0vZly86Q9LM6hlVRRPwpIkZGRHtfjiMpJP1d2XGXRMRb+x5hbXWO\n0wYvJxRrJEOBuX09iBL+b9+sxvw/lTWSK4ALJO1caaWkt0taKunl9Ofby9b9TNJlku4H1gNvSpf9\ns6Rfpk1UP5Y0WtIiSa+kx5hYdowvSXouXbdM0mFdxDEx/Vf7UEmHpMcuTRskPZNuN13SA5LWSnpe\n0lckDUvX/Tw93EPpfh+SdKSklWXn2Sf9DGslPSLphLJ135D0VUk/kfRXSb+S9OauLqykj0h6VtIa\nSfM6rcsa5yhJt0laLeml9PdxXZ3bBg4nFGskrcDPgAs6r5C0C/AT4MvAaOCLwE8kjS7b7CPAHGAH\n4Nl02cx0+R7Am4EHgOuBXYDHgPll+y8FmtN1/wl8T9Lw7gKOiAfS5q+RwCjgQeDb6ep24DxgDHAI\ncBTw8XS/w9Nt9kv3v6nT590G+DFwF/A3wCeBRZLKm8RmAf+UnncFcFmlGCVNBq5Or8PuJNevPAFk\njXMIyTWcAIwHXgO+0t11soHBCcUazSXAJyWN7bT83cCTEfGtiGiLiG8DfwD+Z9k234iIR9L1G9Nl\n10fEHyPiZWAx8MeIuDsi2oDvAfuXdo6IGyNiTbr//wW2BbLUNL4MrAPmpcdbFhEPpsd7BrgGOKLK\nYx0MjAQuj4g3IuJe4DaSJFJyc0T8Ov0si0iSYSUnA7dFxM8j4nXgYmBTaWXWONNr9IOIWB8RfyVJ\nZNV+LmtgQ+sdgFkWEfGwpNuAC0nuIEp2Z8tdR8mzJHceJc9VOOSfy35/rcL8yNKMpPOBM9JzBbAj\nyb/aeyTpTOBI4OCI2JQuewvJnVQLMILk/8dl1RwvjeG50rFSnT/vf5f9vr78s1Q6VmkmItZJWlMW\ne6Y4JY0ArgRmkNwdAewgqamvDypYsfkOxRrRfOAf6PjluYqkiaXceOD/lc33emjttF7yGeCDwKiI\n2Bl4GVCV+34OODG9Eyq5muQualJE7Aj872qOl1oF7Nnp4YLOn7dazwN7lsU7gqTZq7dxnk9y53ZQ\nun2pWazaz2YNygnFGk5ErABuAs4pW3w78BZJp6TF8A8Bk0magWphB6ANWA0MlXQJyR1KtyTtmcb6\n0Yh4osIxXwFelbQ3cFan9X8G3tTFoX9F0nz2j5K2kXQkSfPed6r7OB18H3iPpHekxfZL6fjdkDXO\nHUju7tamta352KDghGKN6lJgc5+UiFgDvIfkX8drgH8E3hMRL9bofHeS1FieIGla2kDlJrTOjgJ2\nBb5f9qTXI+m6C4BTgL8C15IknnILgBvSp6s+WL4iIt4ATgCOB14EvkaStP6Q9YNFxCPAJ0geNHge\neAlYWbZJ1jivArZL43oQuCNrTNaY5BdsmZlZLfgOxczMasIJxczMasIJxczMasIJxczMamJQdWwc\nM2ZMTJw4sd5hmJk1lGXLlr0YEZ1Hp9jKoEooEydOpLW1td5hmJk1FEmdR6GoyE1eZmZWE04oZmZW\nE04oZmZWE4OqhlLJxo0bWblyJRs2bKh3KA1p+PDhjBs3jm222abeoZhZnQ36hLJy5Up22GEHJk6c\niOTBULOICNasWcPKlSvZa6+96h2OmdXZoG/y2rBhA6NHj3Yy6QVJjB492nd3ZgY4oQA4mfSBr52Z\nlTihmFmXFi2CiRNhyJDk56JF9Y7IiswJpQCamppobm5mypQpfOADH2D9+vWZj3HGGWfw6KOPAvD5\nz3++w7q3v/3tNYnTBpdFi2DOHHj2WYhIfs6Z46RiXRtU70NpaWmJzj3lH3vsMfbZZ586RZQYOXIk\nr776KgCzZ89m2rRpfOpTn6rJ8fpDEa6h1d7EiUkS6WzCBHjmmf6OxupJ0rKIaOlpO9+hZJR3E8Bh\nhx3GihUrAPjiF7/IlClTmDJlCldddRUA69at493vfjf77bcfU6ZM4aabkpfnHXnkkbS2tnLhhRfy\n2muv0dzczOzZs4EkwQB86EMf4vbbb998rtNOO40f/OAHtLe38+lPf5oDDzyQqVOncs0119T2Q1lD\n+tOfsi03c0LJIO8mgLa2NhYvXsy+++7LsmXLuP766/nVr37Fgw8+yLXXXstvf/tb7rjjDnbffXce\neughHn74YWbMmNHhGJdffjnbbbcdy5cvZ1GnwGbOnLk5Ab3xxhvcc889vOtd7+K6665jp512YunS\npSxdupRrr72Wp59+ujYfyhrW+PHZltsWg7X25ISSwbx50Lm8sX59srwvSncULS0tjB8/ntNPP51f\n/OIXvPe972X77bdn5MiRvO9972PJkiXsu+++3H333XzmM59hyZIl7LTTTlWf5/jjj+fee+/l9ddf\nZ/HixRx++OFst9123HXXXXzzm9+kubmZgw46iDVr1vDkk0/27UNZw7vsMhgxouOyESOS5da1wVx7\nGvQdG7PIqwmgdEdRrqva1lve8haWLVvG7bffzkUXXcSxxx7LJZdcUtV5hg8fzpFHHsmdd97JTTfd\nxKxZszaf69/+7d847rjj+vZBbEBJW0yZNy/5b3z8+CSZlJZbZd39w3OgX7u63qFI+g9JL0h6uIv1\nkvRlSSsk/U7SAWXrTpX0ZDqd2h/x9mcTwOGHH84tt9zC+vXrWbduHT/84Q857LDDWLVqFSNGjODD\nH/4wF1xwAb/5zW+22nebbbZh48aNFY87c+ZMrr/+epYsWbI5gRx33HFcffXVm/d54oknWLduXe0/\nlDWc2bOTAvymTcnPgf6FWAuDufZU7yavbwAzull/PDApneYAVwNI2gWYDxwETAfmSxqVa6T0bxPA\nAQccwGmnncb06dM56KCDOOOMM9h///35/e9/z/Tp02lubuayyy7js5/97Fb7zpkzh6lTp24uypc7\n9thj+fnPf87RRx/NsGHDgOSR48mTJ3PAAQcwZcoUzjzzTNra2mr/ocwGgUFde4qIuk7ARODhLtZd\nA8wqm38c2A2YBVzT1XZdTdOmTYvOHn300a2WdefGGyMmTIiQkp833php9wEp6zU0y6qR/r+78caI\nESMikgpKMo0YUbuY63EtgNao4vu86DWUPYDnyuZXpsu6Wr4VSXNI7m4YX4N/Isye7dt+s/5UKnKX\n6hKlIjcU8//FPGtPRb8W9W7y6kmlgaKim+VbL4xYGBEtEdEydmyPr0Q2s4LJ6+nKPOVVeyr6tSh6\nQlkJ7Fk2Pw5Y1c1yMxtgBnORu7OiX4uiJ5RbgY+mT3sdDLwcEc8DdwLHShqVFuOPTZeZ2QAzqIvc\nnRT9WtT7seFvAw8Ab5W0UtLpkj4m6WPpJrcDTwErgGuBjwNExF+AzwFL0+nSdJmZ9aDRenEPhg6W\n1f5NCn8tqqncD5SpFk952dZ8DRtH3k8g5aWRnvLKKuvfpMhPeRW9yWtQkMT555+/ef4LX/gCCxYs\nqPl5PKy9Fb2o25WB3MEy69+kyNfCCSWriO7ne2Hbbbfl5ptv5sUXX+zzsbrTOaH88pe/zPV8VjxF\nL+oORgPpb+KEksWCBXDeeVuSSEQy38e7iaFDhzJnzhyuvPLKrdatXr2a97///Rx44IEceOCB3H//\n/ZuXH3PMMRxwwAGceeaZTJgwYXNCOumkk5g2bRpve9vbWLhwIYCHtTeg+EXdwWhA/U2qaRcbKFOf\naiibNkXMnZs0cM6dW3m+l7bffvt4+eWXY8KECbF27dq44oorYv78+RERMWvWrFiyZElERDz77LOx\n9957R0TEJz7xifj85z8fERGLFy8OIFavXh0REWvWrImIiPXr18fb3va2ePHFFzefp/N5IyJuvvnm\n+OhHPxoREa+//nqMGzcu1q9fH9dcc0187nOfi4iIDRs2xLRp0+Kpp57aKn7XUBpHnjWUs86KaGpK\njtnUlMzXSqPVULLEm+ffpFbXjSprKHX/ku/Pqc9F+fIkUpr6mEwitnyxX3zxxXHppZd2SChjx46N\n/fbbb/O0++67xyuvvBL77bdfhy/3UaNGbU4o8+fPj6lTp8bUqVNjxx13jAceeKDDeTqf97XXXotx\n48bFhg0b4pZbbolTTjklIiLe//73x6RJkzafe+LEiXHnnXduFb8TSmPJ48v5rLM6/m9RmmqRVBrt\nQYLexJvH36SW180JJY+EEpEkj/K/UB+TScSWL/Y1a9bEhAkTYsGCBZsTyujRo2P9+vVb7TN16tSK\nCeW+++6LQw89NNatWxcREUcccUTcd999Hc7T+bwRER/+8IfjRz/6UcyaNStuvfXWiIh43/veF3fc\ncUeP8TuhWOnOpPPU1NT3Y0+YUPnYEyb0/dh5KEq8tYyj2oTiGkoWkdZMypXXVPpol1124YMf/CDX\nXXfd5mXHHnssX/nKVzbPl96b8o53vIPvfve7ANx111289NJLALz88suMGjWKESNG8Ic//IEHH3xw\n874e1t7y0t6ebXkWjVa0Lkq89YjDCaVapWTypS/B3LnJM3tz5ybzNUwq559/foenvb785S/T2trK\n1KlTmTx5Mv/+7/8OwPz587nrrrs44IADWLx4Mbvtths77LADM2bMoK2tjalTp3LxxRdz8MEHbz6W\nh7W3vDQ1ZVueRZ5F6yydPKvdtihF9rrEUc1tzECZ+tzkNX9+x5pJqaaSNk/1pw0bNsTGjRsjIuKX\nv/xl7Lfffv0eQ4mbvOyooyo3rxx1VN+PnVcNJctx89o2T66hFD2hRGxdM6lBDaU3nnjiiWhubo6p\nU6dGS0tL/PrXv65LHBFOKJZ/3SCPonWWmLN+vqI8ldbfT3kp2XZwaGlpidbW1g7LHnvsMfbZZ586\nRTQw+BrakCGVW32lpHW4iLLE3Iifr5YkLYuIlp62cw2F5C7NesfXzqA4dYMsssTciJ+vHgZ9Qhk+\nfDhr1qzxF2MvRARr1qxh+PDh9Q6lpvIcjTePInCePv5xGDo0+Zf40KHJfCW9GQU3r8+Xx8i9hR/l\ntyCK/grg3I0bN46VK1eyevXqeofSkIYPH864cePqHUbN5PmK1SzHLsKrXj/+cbj66i3z7e1b5r/2\ntY7bZn3tbV6fL8txs8Sc52t9B5JBX0MxKzdxYvIl1NmECcnIrv117DzjqNbQoZX7kTQ1QV+fHs/r\n8xXhug1E1dZQnFDMyuRZfG20IrDU9bq+fm3k9fmKcN0GooYoykuaIelxSSskXVhh/ZWSlqfTE5LW\nlq1rL1t3a/9GbgNVnsXXRisCN2JnxSJct8GsbglFUhPwVeB4YDIwS9Lk8m0i4ryIaI6IZuDfgJvL\nVr9WWhcRJ/Rb4DagZS2+ZiksN1oRuFR7qHZ5Fnld5zyvW6M9UFEX1XRWyWMCDgHuLJu/CLiom+1/\nCRxTNv9q1nNW6tho1lm1ncHyHlW2CJ3jijAkfRFekduIPeVriaJ3bJR0MjAjIs5I5z8CHBQRZ1fY\ndgLwIDAuItrTZW3AcqANuDwibuniPHOAOQDjx4+f9mylip1ZL7gA3D+KcJ0b7YGKWmuEGkqlkl9X\n2W0m8P1SMkmNTz/gKcBVkt5caceIWBgRLRHRMnbs2L5FbFamKKPKDnRFuM5ZYihCvPVSz4SyEtiz\nbH4csKqLbWcC3y5fEBGr0p9PAT8D9q99iGZdcwG4fxThOjfaAxX1Us+EshSYJGkvScNIksZWT2tJ\neiswCnigbNkoSdumv48BDgUe7ZeozVJFKJw3srwK7XkUxBvtgYq6qabQktcEvAt4AvgjMC9ddilw\nQtk2C0hqJOX7vR34PfBQ+vP0as7norzVWhEK540or0J7Ud7PPtD+u6DoRfl6cMdGs2JwT/nG0ghF\neTMbpPIqXA/mgngROKGY9aMs7fvVjvSbZwx5cU/5gckJxayflEbCffbZpHW/NBJupS/00ki/pcEZ\nSyP99jWpZIkhT3kVrgd1QbwAXEMx6ydZ2vfzGum3SDWGRYvyGQ4+r+MOZh5tuAInFKunLCPh5jXS\nr0fjtd5wUd6sYLK07+c10q9rDJYnJxSzfnLZZTBsWMdlw4ZVbt/POtJvEUbjNXNCMetHnZubumq+\nOvTQpI5SbujQZHlnWQrts2fDwoVJzURKfi5c6BqD1YZrKGb9JK8Ra4tUaLeByTUUs4LJa8Rad+az\nonBCMesneY1Y60K7FYUTilkfZOnNnteItZddBtts03HZNtv4tbdWB9WMIDlQJo82bLV01lkdR7Ut\nTd29JjePEWtvvDFi2LCOMQwb1vcRbgf7a29tCzza8NZclLdayqs3e1ZFGLnXDwYMbC7Km+WsUjLp\nbnleijByrx8MMKhzQpE0Q9LjklZIurDC+tMkrZa0PJ3OKFt3qqQn0+nU/o3cLL/e7FkVYeRePxhg\nUMeEIqkJ+CpwPDAZmCVpcoVNb4qI5nT6errvLsB84CBgOjBf0qh+Ct0Kot5F4Ky92SG/19NW2wM/\n63H92lvLop53KNOBFRHxVES8AXwHOLHKfY8DfhoRf4mIl4CfAjNyitMKqAjDsH/ta3DWWVvuSJqa\nkvmvfa3/Y662B34WWXrVuwe+QR17yks6GZgREWek8x8BDoqIs8u2OQ34F2A1ybvnz4uI5yRdAAyP\niH9Ot7sYeC0ivtDdOV2UHzgasQhchOK5WW80QlG+0gDdnbPbj4GJETEVuBu4IcO+yYbSHEmtklpX\nr17d62CtWBqxCFyE4rlZnuqZUFYCe5bNjwNWlW8QEWsi4vV09lpgWrX7lh1jYUS0RETL2LFjaxK4\n1V9RisBZaiJFKJ6b5ameCWUpMEnSXpKGATOBW8s3kLRb2ewJwGPp73cCx0oalRbjj02X2SBRhCJw\n1pqIX3trA141vR/zmoB3kdRG/gjMS5ddCpyQ/v4vwCPAQ8B9wN5l+/49sCKd/lc153NP+YElS6/z\nPEyYULmn/IQJXe+TV8z1vhY2sOGe8ltzUd5qya/TtcGiEYryZg3NtQuzjpxQzHopz1F+zRqRE4pZ\nH0jdz5sNJk4oZr00bx688UbHZW+8kSw3G4ycUMx6yR0KzTpyQjHrJRflzTpyQjHrpd50KKz3CMlm\neXJCMeulrCPsFmGEZLM8uWOjWT/xqMDWqNyx0axgXMS3gc4JxayfuIhvA50TiuXOhehE1iK+r5s1\nmqH1DsAGtlIhev36ZL5UiIbB93rY0uedNy9p5ho/Pkkmla6Dr5s1IhflLVcuRPeOr5sVSZ+L8pL2\nlfSgpOckLUxfZFVa9+taBWoDmwvRvePrZo2ouxrK1cACYF+Sl2D9QtKb03XbdLWTWTkXonvH180a\nUXcJZWRE3BERayPiC8DZwB2SDgZq0k4maYakxyWtkHRhhfWfkvSopN9JukfShLJ17ZKWp9Otnfe1\nYvDraXvH180aUXcJRZJ2Ks1ExH3A+4FvARO63KtKkpqArwLHA5OBWZImd9rst0BLREwFvg/8n7J1\nr0VEczqd0Nd4LB9Ze5NbwtfNGlGXRXlJpwBPRcSDnZaPBy6OiH/o04mlQ4AFEXFcOn8RQET8Sxfb\n7w98JSIOTedfjYiRWc7poryZWXZ9LspHxH92Tibp8j/1NZmk9gCeK5tfmS7ryunA4rL54ZJa0wcH\nTupqJ0lz0u1aV69e3beIzcysS/Xs2Fjp3XYVb5ckfRhoAa4oWzw+zZinAFeVPTDQ8YARCyOiJSJa\nxo4d29eYjewd7txBz2xwqGfHxpXAnmXz44BVnTeSdDQwDzgiIl4vLY+IVenPpyT9DNgf+GOeAVv2\nDnfuoGc2eNStY6OkoSSPIx8F/D9gKXBKRDxSts3+JMX4GRHxZNnyUcD6iHhd0hjgAeDEiHi0u3O6\nhtJ3WTvcuYOeWeOr2WjDkt6SPrL7cDo/VdJn+xpgRLSRPIp8J/AY8N2IeETSpZJKT21dAYwEvtfp\n8eB9gFZJDwH3AZf3lEysNrJ2uHMHPbPBo8c7FEn/BXwauCYi9k+XPRwRU/ohvpryHUrf+Q7FbPCp\n5ftQRkRE56FW2noXljW6rB3u3EHPbPCoJqG8mD5BFQCSTgaezzUqK6ysHe7cQc9s8KimyetNwELg\n7cBLwNPA7Iio0JBRbG7yMjPLrtomr24fG5Y0hGTok6MlbQ8MiYi/1ipIMzMbOLpt8oqITSRPYhER\n65xMzMysK9XUUH4q6QJJe0rapTTlHpmZmTWUanrK/3368xNlywJ4U+3DMTOzRtVjQomIvfojEDMz\na2w9JhRJH620PCK+WftwzMysUVVTQzmwbDqM5LXAfqGV5cIjE5s1rmqavD5ZPp++xfFbuUVkg5ZH\nJjZrbL15H8p6YFKtAzGbN29LMilZvz5ZbmbFV00N5cdsefHVEJL3v38vz6BscPLIxGaNrZrHhr9Q\n9nsb8GxErMwpHhvExo+vPDLx+PH9H4uZZVdNk9e7IuK/0un+iFgp6V9zj8wGHY9MbNbYqkkox1RY\ndnytAzHzyMRmja3LhCLpLEm/B94q6Xdl09PA72pxckkzJD0uaYWkCyus31bSTen6X0maWLbuonT5\n45KOq0U8Vn+zZycv3tq0KfnpZGLWOLqrofwnsBj4F6D8y/6vEfGXvp5YUhPwVZI7oJXAUkm3dnqV\n7+nASxHxd5JmAv8KfEjSZGAm8DZgd+BuSW+JiPa+xmVmZr3T5R1KRLwcEc9ExKz03SevkTztNVJS\nLcqk04EVEfFURLwBfAc4sdM2JwI3pL9/HzhKktLl34mI1yPiaWBFejwzM6uTHmsokv6npCdJXqz1\nX8AzJHcufbUH8FzZ/Mp0WcVtIqINeBkYXeW+AEiaI6lVUuvq1atrELaZmVVSTVH+n4GDgSfSgSKP\nAu6vwblVYVnn10d2tU01+yYLIxZGREtEtIwdOzZjiGZmVq1qEsrGiFgDDJE0JCLuA5prcO6VwJ5l\n8+OAVV1tI2kosBPwlyr3NTOzflRNQlkraSSwBFgk6UskHRz7aikwSdJekoaRFNlv7bTNrcCp6e8n\nA/dGRKTLZ6ZPge1FMhTMr2sQk5mZ9VI1PeVPJCnInwvMJrlLuLSvJ46INklnA3cCTcB/RMQjki4F\nWiPiVuA64FuSVpDcmcxM931E0neBR0mS2yf8hJeZWX0p+Qd/DxtJE4BJEXG3pBFAUyO+X76lpSVa\nW1vrHYaZWUORtCwiWnrarpqnvP6B5JHda9JFewC39C08MzMbaKqpoXwCOBR4BSAingT+Js+gzMys\n8VSTUF5POx4Cm5+26rmdzMzMBpVqEsp/SfrfwHaSjiF5F8qP8w3LzMwaTTUJ5UJgNfB74EzgduCz\neQZlZmaNp8vHhiWNj4g/RcQm4Np0MjMzq6i7O5TNT3JJ+kE/xGJmZg2su4RSPl7Wm/IOxMzMGlt3\nCSW6+N3MzGwr3Q29sp+kV0juVLZLfyedj4jYMffozMysYXSZUCKiqT8DMTOzxlbNY8NmZmY9ckIx\nM7OacEIxM7OacEIxM7OacEIxM7OaqEtCkbSLpJ9KejL9OarCNs2SHpD0iKTfSfpQ2bpvSHpa0vJ0\nqsU77s3MrA/qdYdyIXBPREwC7knnO1sPfDQi3gbMAK6StHPZ+k9HRHM6Lc8/ZDMz6069EsqJwA3p\n7zcAJ3XeICKeSF/mRUSsAl4AxvZbhGZmlkm9EsrfRsTzAOnPbt8AKWk6MAz4Y9niy9KmsCslbdvN\nvnMktUpqXb16dS1iNzOzCnJLKJLulvRwhenEjMfZDfgW8L/SofQBLgL2Bg4EdgE+09X+EbEwIloi\nomXsWN/gmJnlpbuxvPokIo7uap2kP0vaLSKeTxPGC11styPwE+CzEfFg2bGfT399XdL1wAU1DN3M\nzHqhXk1etwKnpr+fCvyo8waShgE/BL4ZEd/rtG639KdI6i8P5xqtmZn1qF4J5XLgGElPAsek80hq\nkfT1dJsPAocDp1V4PHiRpN+TvJZ4DPDP/Ru+mZl1pojB86qTlpaWaG1trXcYZmYNRdKyiGjpaTv3\nlDczs5pwQjEzs5pwQjEzs5pwQjEzs5pwQjEzs5pwQjEzs5pwQjEzs5pwQjEzs5pwQjEzs5pwQjEz\ns5pwQjEzs5pwQrFE5zHd6jXGW1HiMLPMnFAMFiyA887b8uUdkcwvWDA44zCzXnFCGewiYO1a+NKX\ntnyZn3deMr92bf/dIRQlDjPrNQ9fbx2/vEvmzoUrrwRp8MVhZh1UO3x9XRKKpF2Am4CJwDPAByPi\npQrbtZO8RAvgTxFxQrp8L+A7JO+T/w3wkYh4o6fzOqF0IwKGlN2wbtpUny/xosRhZpsV/X0oFwL3\nRMQk4J50vpLXIqI5nU4oW/6vwJXp/i8Bp+cb7gBXujMoV17LGGxxmFmv1CuhnAjckP5+A8l74auS\nvkf+ncD3e7O/dVLezDR3bnJHMHdux1rGYIrDzHptaJ3O+7cR8TxARDwv6W+62G64pFagDbg8Im4B\nRgNrI6It3WYlsEdXJ5I0B5gDMH78+FrFP3BIsPPOHWsVV16ZrNt55/5rbipKHGbWa7nVUCTdDexa\nYdU84IaI2Lls25ciYlSFY+weEaskvQm4FzgKeAV4ICL+Lt1mT+D2iNi3p5hcQ+lGRMcv7c7zgy0O\nM9us2hpKbncoEXF0V+sk/VnSbundyW7AC10cY1X68ylJPwP2B34A7CxpaHqXMg5YVfMPMNh0/tIe\nzF/imzZt/WDAkDq0Dju5WoOpVw3lVuDU9PdTgR913kDSKEnbpr+PAQ4FHo3kluo+4OTu9rcGVISO\njUceCdOmJUkEkp/TpiXL+1NhgDpsAAAOQUlEQVQRroVZRvVKKJcDx0h6EjgmnUdSi6Svp9vsA7RK\neogkgVweEY+m6z4DfErSCpKaynX9Gr3VXhE6Nm7aBC+/DMuXb0kq06Yl8y+/vCXJ5K0I18KsF9yx\n0YqjCB0by5NISXMzLFvWv81eRbgWZqlCd2ysFyeUBlCEjo2bNkFT05b59vb61VDqfS3MKH7HRusP\nWUbu7dycU8vmnfb27udLIuDcczsuO/fc/m3iKd2hlCuvqfQXd/K0BuSEMlBlKermWYieOBF23XVL\nEmlvT+YnTuy4XQQccgh8+ctwzjlJDOeck8wfckj/1VBKzV3NzUmszc0dayr9wZ08rUE5oQxEWYq6\neRai29th3Tp48cUtSWXXXZP5deu6vlOplyFDYKedOtZMli1L5nfaqf+avbrq5Dl3rjt5WqG5hjJQ\nZSnq5lmILk8iJWPGwH//d8c6RSnmc89N7kpKzjkHrrqqf79E3Q/FrAMX5SsYVAkFshV18yxEt7fD\n0LI+tG1tWyeTEheizQrHRfmBqtpCe5aibtZCdJYCfukOpVx5TaVzzFmK8lniKMoDCln4dcjWYJxQ\nGkm1hfYsRd2shegsBfz2dthxx6S5a8yY5M5kzJhkfscdOyaViKROUakov9NOW3+ZZomjKA8oZOGe\n8taAnFAaRZZCe5aibpZCdNYC/pAhW5q2Zs5M5mfOTOabmrY+dls6gPSiRcn8okXJfFtbx2NniaMo\nDyhk4Z7y1qgiYtBM06ZNi4a2aVPE3LkRyVdKMs2dmyzvavvu5su1t3c/X768ubljDM3NXW+/aVPE\n2Wd33P7ssyvH0tYWMXp0x21Hj06W9yWOLNct6+fLS9a/tVmOgNao4jvWRfm85PWETpYCdxZZnmxq\na4Ntttkyv3Fjx5g6yxLzxo0wbNiW+Tfe6Hiu3h43rwcU8vo7l47lBxSsAFyUr6e82r+POKJygfuI\nI/p23Cx1gyOOgB126Lhshx26jiFLzIcdBttt13HZdtslyzu75JLKx73kkq23LV3/crV4QCHPOkeW\nmM2KoprbmIEy9UuTV3lTRamJovN8b7S1RYwZkxxnzJjK871R3sRTatrpPF+ycWPE8OHJuuHDK8/3\nNuY33ohoakrWNTVVnu/NcbP8PbJci7z+znkf26wXqLLJq+5f8v059VsNJa/274sv3vLFWZrGjEmW\n90WWusFhh21JIqVp+PBkeV9jHj9+SxIpTU1NyfLODj+88nEPP3zrbefP73j9S3+f+fO33vaIIzp+\n9tK1OeKIrbfNs86RJWaznFWbUFxDyUvk1P6dZw2l2rpBo9ZQqq11ZKkn5fV3rhRjdzGb5cg1lHqK\nSB7RLTd3bvedELubL9m0CVo6/U1bWrp+nDXLcautG2zaBAce2HHZgQd23wmy2pg3bYLp0zsumz69\n8rYRcP75HZedf37XnzHLK46r3TYi39GR/VpmazTV3MbUegJ2AX4KPJn+HFVhm/8BLC+bNgAnpeu+\nATxdtq65mvP2Ww1ljz2S5o9PfjKZ/+Qnk/k99ti6OaTapo0s7ft5HTdrDO3tEbvuWnn7XXft/bHz\nrDFUe902bYo46KDknOeck8yfc04yf9BBrnPYgEKVTV71ukO5ELgnIiYB96TzHUTEfRHRHBHNwDuB\n9cBdZZt8urQ+IpZ33r9uIrb0AF+yJJlfsiSZb2/v+K/XiOo7sGXpgJjXcbOOxittaYI67LBkvvTE\nVlNT7ztY5jUab5brZmZbqybr1HoCHgd2S3/fDXi8h+3nAIvK5r8BnJz1vP1WlM+r013p2N3N533c\nrNuW36GVptKdWy2O3d18b2S5buV3JaWpdLdiNoBQ5KK8pLURsXPZ/EsRMaqb7e8FvhgRt6Xz3wAO\nAV4nvcOJiNe72HcOSUJi/Pjx05599tmafY5uZe0cl0dhN6/jNmoc1coSb6N9NrNeqHtRXtLdkh6u\nMJ2Y8Ti7AfsCd5YtvgjYGziQpB7zma72j4iFEdESES1jx47txSfphSxF7lKzSrladGDL67i9iaPe\nr/XNIst1K8o1LsXS3bxZf6jmNqbWExmavIC5wMJu1h8J3FbNefulyasIxeWidIxrtMJ1lutWlGsc\n4T4rljuqbPLqpvNArm4FTgUuT3/+qJttZ5HckWwmabeIeF6SgJOAh/MKNLOuisvTplVfXIa+FZfz\nOu5Al+W6FeUaR9mDBJDEUP7qgnDfFes/9aqhjAa+C4wH/gR8ICL+IqkF+FhEnJFuNxG4H9gzIjaV\n7X8vMBYQyWPDH4uIV3s6b792bMzaOS6PDmx5HTdrDEV4rW8WWa5bUa5xta97NusFvwK4gkH3CuCi\ncOE6f77GlqO6F+UHHBc9e6dIheuBytfYCsIJpRp+HWvvlDfF9PQqYusdX2MrkHoV5RuHi569V5TC\n9UDma2wF4hpKNVz07JsiFK4HOl9jy5GL8hX0qShflKKnvzjMrJ+5KF9LRSl6upZjZgXmhNKTohQ9\ny2s5HgnXzArIRfmeFKXoWX7eL31pSz3HtRwzKwjXUKpVlNpFUWo5ZjZouIZSa0V4HWtRajlmZhU4\noTSKotRyzMy64BpKoyhKLcfMrAuuoTSaotRyzGzQcA1loCpCLcfMrAInFDMzqwknFDMzq4m6JBRJ\nH5D0iKRN6Vsau9puhqTHJa2QdGHZ8r0k/UrSk5JukjSsfyI3M7Ou1OsO5WHgfcDPu9pAUhPwVeB4\nYDIwS9LkdPW/AldGxCTgJeD0fMM1M7Oe1CWhRMRjEfF4D5tNB1ZExFMR8QbwHeBESQLeCXw/3e4G\n4KT8ojUzs2oUuYayB/Bc2fzKdNloYG1EtHVaXpGkOZJaJbWuXr06t2DNzAa73Do2Srob2LXCqnkR\n8aNqDlFhWXSzvKKIWAgsTGNaLenZKs5dMgZ4McP2RdCIMUNjxt2IMUNjxt2IMUNjxl0p5gnV7Jhb\nQomIo/t4iJXAnmXz44BVJB90Z0lD07uU0vJqYhqbJQBJrdV05imSRowZGjPuRowZGjPuRowZGjPu\nvsRc5CavpcCk9ImuYcBM4NZIuvbfB5ycbncqUM0dj5mZ5ahejw2/V9JK4BDgJ5LuTJfvLul2gPTu\n42zgTuAx4LsR8Uh6iM8An5K0gqSmcl1/fwYzM+uoLoNDRsQPgR9WWL4KeFfZ/O3A7RW2e4rkKbC8\nLeyHc9RaI8YMjRl3I8YMjRl3I8YMjRl3r2MeVINDmplZfopcQzEzswbihGJmZjXhhNINSZ+T9DtJ\nyyXdJWn3esdUDUlXSPpDGvsPJe1c75h6Uu34bkXR1ThzRSbpPyS9IOnhesdSLUl7SrpP0mPpfx9z\n6x1TTyQNl/RrSQ+lMf9TvWPKQlKTpN9Kui3rvk4o3bsiIqZGRDNwG3BJvQOq0k+BKRExFXgCuKjO\n8VSjx/HdiqKHceaK7BvAjHoHkVEbcH5E7AMcDHyiAa7168A7I2I/oBmYIengOseUxVySJ2szc0Lp\nRkS8Uja7Pd30yC+SiLirbGiaB0k6fxZaleO7FUXFcebqHFOPIuLnwF/qHUcWEfF8RPwm/f2vJF90\nXQ61VASReDWd3SadGuK7Q9I44N3A13uzvxNKDyRdJuk5YDaNc4dS7u+BxfUOYoDpapw5y5GkicD+\nwK/qG0nP0maj5cALwE8jovAxp64C/hHY1JudB31CkXS3pIcrTCcCRMS8iNgTWETS0bIQeoo73WYe\nSZPBovpFukU1MTeITOPJWd9JGgn8ADi3U8tBIUVEe9pUPg6YLmlKvWPqiaT3AC9ExLLeHqMuHRuL\nJMOYY/8J/ASYn2M4VespbkmnAu8BjoqCdDaqwfhuRdHVOHOWA0nbkCSTRRFxc73jySIi1kr6GUnt\nqugPQxwKnCDpXcBwYEdJN0bEh6s9wKC/Q+mOpEllsycAf6hXLFlImkEyPM0JEbG+3vEMQBXHmatz\nTANS+v6j64DHIuKL9Y6nGpLGlp6slLQdcDQN8N0RERdFxLiImEjy3/S9WZIJOKH05PK0SeZ3wLEk\nTz80gq8AOwA/TR95/vd6B9STrsZ3K6IexpkrLEnfBh4A3ipppaRGeNPpocBHgHem/y0vT/8FXWS7\nAfel3xtLSWoomR/BbUQeesXMzGrCdyhmZlYTTihmZlYTTihmZlYTTihmZlYTTihmZlYTTihmVZLU\nXvbo6vJ0KJCsx9hZ0sdrH93m4+8t6QFJr0u6IK/zmFXix4bNqiTp1YgY2cdjTARui4hMQ3FIaoqI\n9iq2+xtgAnAS8FJEfKE3cZr1hu9QzPogHQTwCklL0/fPnJkuHynpHkm/kfT7svHKLgfenN7hXCHp\nyPL3Tkj6iqTT0t+fkXSJpF8AH5D0Zkl3SFomaYmkvTvHExEvRMRSYGPuH96sk0E/lpdZBtulI8gC\nPB0R7wVOB16OiAMlbQvcL+kuktGI3xsRr0gaAzwo6VbgQpJ31TQDSDqyh3NuiIh3pNveA3wsIp6U\ndBDwNeCdtf6QZr3lhGJWvddKiaDMscBUSSen8zsBk0gGkPy8pMNJhgLfA/jbXpzzJtg82u7bge8l\nw1sBsG0vjmeWGycUs74R8MmI6DD2WNpsNRaYFhEbJT1DMoJrZ210bHruvM269OcQYG2FhGZWGK6h\nmPXNncBZ6RDrSHqLpO1J7lReSJPJ/yAplAP8lWTgzpJngcmStpW0E3BUpZOk7wB5WtIH0vNI0n75\nfCSz3vEdilnffB2YCPwmHWp9NckTVouAH0tqBZaTDl8eEWsk3S/pYWBxRHxa0neB3wFPAr/t5lyz\ngaslfZbktbLfAR4q30DSrkArsCOwSdK5wORGeCmVNT4/NmxmZjXhJi8zM6sJJxQzM6sJJxQzM6sJ\nJxQzM6sJJxQzM6sJJxQzM6sJJxQzM6uJ/w+50DH8YxKlIwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb964320>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 均值\n",
    "u = np.mean(X, axis=0)\n",
    "# 方差\n",
    "v = np.var(X, axis=0)\n",
    "\n",
    "X = (X - u) / v\n",
    "\n",
    "# 作图\n",
    "plt.scatter(X[:50, 0], X[:50, 1], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 0], X[50:, 1], color='red', marker='x', label='Negative')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.title('Normalization data')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 初始化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# X加上偏置项\n",
    "X = np.hstack((np.ones((X.shape[0],1)), X))\n",
    "# 权重初始化\n",
    "W = np.random.randn(3,1)\n",
    "N = 1000     # 迭代次数\n",
    "lr = 0.2    # 学习因子"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 迭代更新训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl4XfV95/H3R1ebtdmyLdvgBRsw\nJgaCIcIEyEJSAk4XnKS0gbSN0+QZQgeatGmaIdM8yQwdZtKkzdIOTfAkpNM+ISQkTXAZJ26aAFkB\ny9gsBgy2wba8yrZkWdYufeePeySuZVnyouOr5fN6nvvce37nd66+R9f2x7/zO/ccRQRmZmZDKch3\nAWZmNvo5LMzMbFgOCzMzG5bDwszMhuWwMDOzYTkszMxsWKmGhaRlkjZJ2izpziH63SQpJNUmy/Ml\ntUnakDy+mmadZmY2tMK03lhSBrgHeAdQD6yVtCoinh/QrxL4CPDEgLfYEhFL0qrPzMxOXJoji6XA\n5ojYGhGdwAPA8kH6/TXwOaA9xVrMzOw0pDayAGYDO3KW64ErcztIugyYGxEPS/r4gO0XSFoPNAOf\nioifD/wBkm4FbgUoLy9/w4UXXjiS9ZuZjXvr1q3bHxE1w/VLMyw0SFv/tUUkFQBfBD4wSL/dwLyI\nOCDpDcAPJF0UEc1HvVnESmAlQG1tbdTV1Y1U7WZmE4KkbSfSL83DUPXA3JzlOcCunOVK4GLgUUmv\nAm8EVkmqjYiOiDgAEBHrgC3ABSnWamZmQ0gzLNYCCyUtkFQM3Ays6lsZEYciYnpEzI+I+cDjwI0R\nUSepJpkgR9K5wEJga4q1mpnZEFI7DBUR3ZLuANYAGeC+iNgo6S6gLiJWDbH5W4C7JHUDPcBtEXEw\nrVrNzGxoGi+XKPechZnZyZO0LiJqh+vnb3CbmdmwHBZmZjYsh4WZmQ1rwodFS0c3X/j3Tazf3pjv\nUszMRq0JHxad3b38/U838/SOpnyXYmY2ak34sCguzP4KunrGx1lhZmZpmPBhUZTJXpWks6c3z5WY\nmY1eEz4sijPZX0FHt8PCzOx4JnxYSKI4U0Cnw8LM7LgmfFhA9lBUlw9DmZkdl8OC7CS3RxZmZsfn\nsMBhYWY2HIcFSVj4MJSZ2XE5LICijMPCzGwoDgvw2VBmZsNwWAAlnrMwMxtSqmEhaZmkTZI2S7pz\niH43SQpJtTltn0y22yTphjTrLMoU+NRZM7MhpHZb1eQe2vcA7wDqgbWSVkXE8wP6VQIfAZ7IaVtM\n9p7dFwFnA/8h6YKI6EmjVp8NZWY2tDRHFkuBzRGxNSI6gQeA5YP0+2vgc0B7Ttty4IGI6IiIV4DN\nyfulwmdDmZkNLc2wmA3syFmuT9r6SboMmBsRD5/stsn2t0qqk1TX0NBwyoV6gtvMbGhphoUGaeu/\nDrikAuCLwF+c7Lb9DRErI6I2ImprampOudAijyzMzIaU2pwF2dHA3JzlOcCunOVK4GLgUUkAs4BV\nkm48gW1HVIlHFmZmQ0pzZLEWWChpgaRishPWq/pWRsShiJgeEfMjYj7wOHBjRNQl/W6WVCJpAbAQ\neDKtQj3BbWY2tNRGFhHRLekOYA2QAe6LiI2S7gLqImLVENtulPQd4HmgG7g9rTOhwKfOmpkNJ83D\nUETEamD1gLZPH6fvtQOW7wbuTq24HB5ZmJkNzd/gxqfOmpkNx2FB9tTZrp6gt/eYE67MzAyHBZAd\nWQAeXZiZHYfDAigtygDQ0eWwMDMbjMMCKC3K/hrau1M74crMbExzWACTkpFFW6fDwsxsMA4LXjsM\n5ZGFmdngHBbkHIbynIWZ2aAcFrw2svBhKDOzwTks8GEoM7PhOCyA0sIkLDyyMDMblMMCmFTskYWZ\n2VAcFniC28xsOA4L/D0LM7PhOCzwBLeZ2XBSDQtJyyRtkrRZ0p2DrL9N0rOSNkj6haTFSft8SW1J\n+wZJX02zzpJCH4YyMxtKajc/kpQB7gHeQfae2mslrYqI53O63R8RX0363wh8AViWrNsSEUvSqm9A\nrZQWFdDe5ZGFmdlg0hxZLAU2R8TWiOgEHgCW53aIiOacxXIgbzeUKC3KOCzMzI4jzbCYDezIWa5P\n2o4i6XZJW4DPAR/JWbVA0npJj0l682A/QNKtkuok1TU0NJxWsaWFGU9wm5kdR5phoUHajhk5RMQ9\nEXEe8F+ATyXNu4F5EXEZ8DHgfklVg2y7MiJqI6K2pqbmtIotK87Q6pGFmdmg0gyLemBuzvIcYNcQ\n/R8A3gUQER0RcSB5vQ7YAlyQUp0AlJcUcqSjO80fYWY2ZqUZFmuBhZIWSCoGbgZW5XaQtDBn8beA\nl5P2mmSCHEnnAguBrSnWSoXDwszsuFI7GyoiuiXdAawBMsB9EbFR0l1AXUSsAu6QdB3QBTQCK5LN\n3wLcJakb6AFui4iDadUK2ZHFzqa2NH+EmdmYlVpYAETEamD1gLZP57z+6HG2+x7wvTRrG6iiJENL\nR9eZ/JFmZmOGv8GdyM5ZeILbzGwwDotERUkhLZ6zMDMblMMiUVFSSGd3L53dvuSHmdlADotEeUl2\n+sZnRJmZHcthkahIwsKHoszMjuWwSPSPLDodFmZmAzksEhWlycii3WFhZjaQwyJRUZK9AZIPQ5mZ\nHcthkXhtgtvftTAzG8hhkSgv9tlQZmbH47BIVCZzFocdFmZmx3BYJPw9CzOz43NYJIoyBRQXFniC\n28xsEA6LHJMnFXGo1VeeNTMbyGGRo7qsiKa2znyXYWY26jgsckwpK6bRIwszs2OkGhaSlknaJGmz\npDsHWX+bpGclbZD0C0mLc9Z9Mtluk6Qb0qyzT3VZEU2tHlmYmQ2UWlgk99C+B3gnsBi4JTcMEvdH\nxCURsQT4HPCFZNvFZO/ZfRGwDPjHvntyp6naIwszs0GlObJYCmyOiK0R0Qk8ACzP7RARzTmL5UAk\nr5cDD0RER0S8AmxO3i9VU8qKOdTaRUQM39nMbAJJMyxmAztyluuTtqNIul3SFrIji4+c5La3SqqT\nVNfQ0HDaBU8pK6Kzp5fWTl/yw8wsV5phoUHajvkve0TcExHnAf8F+NRJbrsyImojorampua0ioXs\nnAVAo+ctzMyOkmZY1ANzc5bnALuG6P8A8K5T3HZETCkrBqDJ8xZmZkdJMyzWAgslLZBUTHbCelVu\nB0kLcxZ/C3g5eb0KuFlSiaQFwELgyRRrBbIT3OCwMDMbqDCtN46Ibkl3AGuADHBfRGyUdBdQFxGr\ngDskXQd0AY3AimTbjZK+AzwPdAO3R0TqEwk+DGVmNrjUwgIgIlYDqwe0fTrn9UeH2PZu4O70qjvW\na4ehHBZmZrn8De4cU8qKkODAEYeFmVkuh0WOokwBU8uK2Xe4I9+lmJmNKg6LAWoqS9jX3J7vMszM\nRhWHxQAzqko9sjAzG8BhMcCMyhL2NTsszMxyOSwGmFFZwv6WDnp7fX0oM7M+DosBZlSW0N0bHPTp\ns2Zm/RwWA8ysKgXwoSgzsxwOiwFmVJUAsPewz4gyM+vjsBhgRmXfyMJhYWbWx2ExwKzJpRQIdja2\n5bsUM7NRw2ExQFGmgLMmT6LeYWFm1s9hMYjZ1Q4LM7NcDotBzKmeRH1ja77LMDMbNRwWg5hTXcae\n5nY6u3vzXYqZ2aiQalhIWiZpk6TNku4cZP3HJD0v6RlJP5F0Ts66HkkbkseqgdumaU71JHoD9hzy\nGVFmZpBiWEjKAPcA7wQWA7dIWjyg23qgNiJeD3wX+FzOuraIWJI8bkyrzsHMqZ4EwA4fijIzA9Id\nWSwFNkfE1ojoBB4Alud2iIhHIqLvX+THgTkp1nPCFkwvB2Dr/iN5rsTMbHRIMyxmAztyluuTtuP5\nEPDDnOVSSXWSHpf0rsE2kHRr0qeuoaHh9CtOzKoqpbw4w5Z9LSP2nmZmY1ma9+DWIG2DXspV0h8C\ntcBbc5rnRcQuSecCP5X0bERsOerNIlYCKwFqa2tH7DKxkjhvRgWbHRZmZkC6I4t6YG7O8hxg18BO\nkq4D/gq4MSL6r94XEbuS563Ao8BlKdZ6jPNrKtjS4LAwM4N0w2ItsFDSAknFwM3AUWc1SboMuJds\nUOzLaa+WVJK8ng5cAzyfYq3HOG9GBbsPtdPS0X0mf6yZ2ag0ZFhIukrSPcmprQ2StktaLel2SZOH\n2jYiuoE7gDXAC8B3ImKjpLsk9Z3d9HmgAnhwwCmyrwPqJD0NPAJ8NiLObFjUVAB43sLMjCHmLCT9\nkOxho4eAu4F9QClwAfA24CFJX4iI434HIiJWA6sHtH065/V1x9nuV8AlJ74bI+/8Gdmw2LyvhUvn\nTslnKWZmeTfUBPcfRcT+AW0twFPJ4++SQ0Tj0jnTyijKiJf2Hc53KWZmeXfcw1B9QSHpg5IWDtVn\nPCrKFLBoViUbdzbnuxQzs7w7kQnu+cC9krZI+o6kP5W0JOW6RoVLZk/m2Z2HiBixs3LNzMakYcMi\nIj4dEW8HLgZ+AfwlsC7twkaDi2dP5lBbly9XbmYT3rBhIelTyWT3vwPnAx9nlFyWI22XzM6e8PXs\nzkN5rsTMLL9O5DDUe4BpwH8A/wqsiojdqVY1SiyaVUlRRg4LM5vwTuQw1OXAbwBPAu8AnpX0i7QL\nGw1KCjMsmlXJM/VN+S7FzCyvhr02lKSLgTeTvW5TLdmLA/485bpGjdpzpvLttTvo6umlKON7RZnZ\nxHQi//r9DVAJ/D3wuoh4W+4X68a7pQum0tbVw3M+FGVmE9iwI4uI+K0zUchodcX8qQA88cpBLptX\nnedqzMzy47gjC0n/Jul3JBUNsu7c5BpPH0y3vPyrqSzh3JpynnzlYL5LMTPLm6FGFv8J+BjwJUkH\ngQZgEtkv6W0G7omIH6Re4Shw5YKpPPz0brp7ein0vIWZTUDHDYuI2AN8AviEpPnALKANeCkiJtS3\n1N68sIZvPbmDp7Y3sXTB1HyXY2Z2xg111dnDHH1nO/UtS+oAtgB/FRE/SbXCUeBNC6dTWCAe2bTP\nYWFmE9JQFxKsjIiqnEf/MtlRxoeBL5+xSvOoqrSI2vnVPPLivuE7m5mNQ6d0AD4ieiLiaeAfRrie\nUevtF87gxT2H2dU0oY7AmZkBp3lb1Yi4d6j1kpZJ2iRps6Q7B1n/MUnPJ3fi+4mkc3LWrZD0cvJY\ncTp1joS3LZoBwE89ujCzCSi1U3skZYB7gHcCi4FbJC0e0G09UBsRrwe+C3wu2XYq8BngSmAp8BlJ\nef2Sw/kzKlgwvZwfPjchLotlZnaUNM8DXQpsjoitEdEJPAAsz+0QEY9ERGuy+DivXc32BuDHEXEw\nIhqBHwPLUqx1WJL4ndefxa+3HGDf4fZ8lmJmdsalGRazyV5Hqk990nY8HwJ+eDLbSrpVUp2kuoaG\nhtMsd3i/c+nZ9AasfsajCzObWNIMCw3SNugt5yT9IdmLFH7+ZLaNiJURURsRtTU1Nadc6IlaOLOS\nC2dV8m8OCzObYNIMi3pgbs7yHGDXwE6SrgP+CrgxIjpOZtt8uHHJ2azb1sjWhpZ8l2JmdsakGRZr\ngYWSFkgqBm4GVuV2kHQZcC/ZoMg9zWgNcL2k6mRi+/qkLe9uesMcCgvEt57cnu9SzMzOmNTCIiK6\ngTvI/iP/AvCdiNiYXIDwxqTb54EK4EFJGyStSrY9CPw12cBZC9yVtOXdjMpSrr9oJt9dV097V0++\nyzEzOyOGvUT56YiI1cDqAW2fznl93RDb3gfcl151p+59S89h9bN7WLNxD8uXDDVnb2Y2PvgSqqfg\n6vOmcc60Mv7l19vyXYqZ2RnhsDgFBQVixVXzqdvWyLpto+LomJlZqhwWp+jmpXOZUlbEVx7dmu9S\nzMxS57A4RWXFhay4aj7/8cJeXt57ON/lmJmlymFxGlZcPZ/SogLueWRzvksxM0uVw+I0TC0vZsVV\n83no6V28sLs53+WYmaXGYXGa/uTa86goKeRv12zKdylmZqlxWJymKWXF3PbW8/jJi/tY+6rPjDKz\n8clhMQI+eM0CZlSWcPf/e4He3kGvlWhmNqY5LEbApOIMf3nDIjbsaOK76+rzXY6Z2YhzWIyQ3718\nDrXnVPPZH71IU2tnvssxMxtRDosRUlAg/vpdF3OorYvPebLbzMYZh8UIet1ZVXzg6vnc/8R2frVl\nf77LMTMbMQ6LEfbx6xexYHo5f/ngMxxu78p3OWZmI8JhMcImFWf4u9+/lN2H2vgfD7+Q73LMzEZE\nqmEhaZmkTZI2S7pzkPVvkfSUpG5JNw1Y15PcEKn/pkhjxeXzqrntrefx7bod/PvGPfkux8zstKUW\nFpIywD3AO4HFwC2SFg/oth34AHD/IG/RFhFLkseNg6wf1T563UIumT2Zjz/4NDsOtua7HDOz05Lm\nyGIpsDkitkZEJ/AAsDy3Q0S8GhHPAL0p1pEXJYUZ7nnf5QRw+/1P0dHtW7Ca2diVZljMBnbkLNcn\nbSeqVFKdpMclvWuwDpJuTfrUNTQ0nE6tqZg3rYy/+71Leab+EHf92/P5LsfM7JSlGRYapO1kroUx\nLyJqgfcBX5J03jFvFrEyImojorampuZU60zV9RfN4sNvPZdvPrGd//urV/NdjpnZKUkzLOqBuTnL\nc4BdJ7pxROxKnrcCjwKXjWRxZ9InbriQ6143g//+bxt57KXRNwIyMxtOmmGxFlgoaYGkYuBm4ITO\napJULakkeT0duAYYs8dxMgXiyzdfxqJZVdzxzad4yXfWM7MxJrWwiIhu4A5gDfAC8J2I2CjpLkk3\nAki6QlI98HvAvZI2Jpu/DqiT9DTwCPDZiBizYQFQXlLI11fUUlqc4f1ff9JnSJnZmKKI8XFJ7dra\n2qirq8t3GcN6cU8z7733caaUFfHgbVcxo7I03yWZ2QQmaV0yPzwkf4P7DLtwVhXf+OMraDjcwfu/\n/iSHWn1JEDMb/RwWeXD5vGr+z/tr2dpwhD/4+uM0HvElzc1sdHNY5Mk150/n3ve/gZf2tnDL/3mc\nhsMd+S7JzOy4HBZ59LZFM/jGB67g1QNHeO/KX7PnUHu+SzIzG5TDIs+uOX86//zBK9l7qJ3f/cqv\neNmn1ZrZKOSwGAWWLpjKtz98FZ09vbznK7/i11sO5LskM7OjOCxGiYtnT+Zf/+RqZlaVsuK+J3lo\nw858l2Rm1s9hMYrMnVrG9267msvmTeGjD2zgsz98kZ7e8fE9GDMb2xwWo8zksiL++UNL+YMr5/HV\nx7aw4r4nfWqtmeWdw2IUKinMcPe7L+Fzv/t6nnzlIL/9D7/guZ2H8l2WmU1gDotR7PevmMuDt11F\nbwTv/sdf8rWfb6XXh6XMLA8cFqPcpXOnsPojb+Zti2bwP/7fC6z4xpPsa/b3MczszHJYjAHV5cXc\n+0dv4H+++xLWvnqQZV/+OWs27sl3WWY2gTgsxghJvO/KeTz8p29iVlUpH/6Xddx+/1O+TIiZnREO\nizHm/BmVPHTHNXz8+gv48ca9vOOLj/H99fWMl0vNm9nolGpYSFomaZOkzZLuHGT9WyQ9Jalb0k0D\n1q2Q9HLyWJFmnWNNUaaAO96+kNUffRPnTi/nz7/9NCu+sZatDS35Ls3MxqnUwkJSBrgHeCewGLhF\n0uIB3bYDHwDuH7DtVOAzwJXAUuAzkqrTqnWsOn9GJQ/edjX/7XcWs35bIzd86Wf8rx++QEtHd75L\nM7NxJs2RxVJgc0RsjYhO4AFgeW6HiHg1Ip4BegdsewPw44g4GBGNwI+BZSnWOmZlCsQHrlnATz9+\nLe9aMpt7H9vK2/72Ub63rt6n2ZrZiEkzLGYDO3KW65O2tLedkGoqS/j8713KD26/hrMnl/IXDz7N\nb/3DL3hk0z7PZ5jZaUszLDRI24n+q3VC20q6VVKdpLqGhoaTKm68WjJ3Ct//z9fw5ZuXcKSjmz/+\nxlpuXvk4T21vzHdpZjaGpRkW9cDcnOU5wK6R3DYiVkZEbUTU1tTUnHKh401BgVi+ZDb/8bG3ctfy\ni9jS0MJ7/vFXfOif1rLeoWFmpyDNsFgLLJS0QFIxcDOw6gS3XQNcL6k6mdi+Pmmzk1BcWMD7r5rP\nY3/5Nj5+/QWs297Iu//xV/zh157g8a0HfHjKzE6Y0vwHQ9JvAl8CMsB9EXG3pLuAuohYJekK4PtA\nNdAO7ImIi5JtPwj81+St7o6Ibwz1s2pra6Ouri6tXRkXjnR0880ntrHyZ6+wv6WD2nOque2t5/H2\nC2dQUDDYkT8zG+8krYuI2mH7jZf/XTosTlx7Vw/fqdvBVx/dwq5D7cyfVsYHrp7PTbVzqSgpzHd5\nZnYGOSxsWF09vfzouT3c98tXWL+9icqSQn7/irmsuGo+86aV5bs8MzsDHBZ2UtZvb+Qbv3yV1c/u\nprs3uOb8adx8xTyuv2gmJYWZfJdnZilxWNgp2X2ojQfr6vn22h3sbGqjuqyI91w+h1uWzuX8GZX5\nLs/MRpjDwk5LT2/wy837eWDtdn78/F66eoJL505h+aVn89uXnsWMytJ8l2hmI8BhYSNmf0sH339q\nJ99fv5PndzdTILjm/OnceOnZ3HDxLKpKi/JdopmdIoeFpeLlvYdZ9fQuHtqwi+0HWykuLODaC2q4\n4aJZ/MbrZjClrDjfJZrZSXBYWKoigg07mnhowy5+9Nwe9jS3kykQbzx3KjdcNIvrF89i1mQfqjIb\n7RwWdsb09gbP7DzEmo17WLNxD1sbjgBw6ZzJXLtoBtcuquH1c6aQ8Rf/zEYdh4XlzeZ9h1mzcS8/\nfn4vT9c3EQHVZUW8eWEN1y6q4c0La6ipLMl3mWaGw8JGicYjnfzs5QYee6mBn73UwP6WTgAunl3F\nVedO46rzplE7f6onyc3yxGFho05vb/D87mYe3bSPn7+8n/Xbm+js6aVAcMnsybzxvGm88dxpXDF/\nqi87YnaGOCxs1Gvv6uGpbY08vvUAv956gA07mujqCTIF4uKzq7hsXjVvOKeay8+p5uzJpUie8zAb\naQ4LG3NaO7t5alsTv966n7pXG3m6von2ruwdd2dVlXL5OVO4fF42PC46u8qXITEbAScaFh7r26hR\nVlzImxZO500LpwPZCx2+uPsw67Yd5KntTTy1vZHVz+4BoCgjFs2q5JLZk7l49mQumT2ZRbMqHSBm\nKfHIwsaUfc3trNvWyIb6Jp7beYjndjZzqK0LgMICccHMJEDmTObis6u4YGYl5Z7/MDsuH4ayCSEi\nqG9s49mdh3h25yGeS56bWrv6+8ybWsaiWZVcOKuy/3n+tHIKM2neKNJsbBgVh6EkLQO+TPZOeV+L\niM8OWF8C/DPwBuAA8N6IeFXSfOAFYFPS9fGIuC3NWm1sksTcqWXMnVrGb15yFvBagGzc1cxLew+z\nac9hXtzTzE9e2Etv8n+j4kwB582o4MJZlSycWcF5NRWcV1POvKnlFBc6RMwGSi0sJGWAe4B3APXA\nWkmrIuL5nG4fAhoj4nxJNwN/A7w3WbclIpakVZ+NX7kBsuziWf3t7V09bN7XwqY9h3lp72Fe3HOY\nX285wPfX7+zvkykQ86aWcV5NOecmAZJ9rmBqua97ZRNXmiOLpcDmiNgKIOkBYDmQGxbLgf+WvP4u\n8L/l8yMtJaVFGS5OJsRzNbd38UrDEbY0tLC14Qhb97ewZd8Rfvbyfjq7e/v7TSkr4pxp5cybWsY5\nU8uYN7WMedPKOGdaGTMrS30fcxvX0gyL2cCOnOV64Mrj9YmIbkmHgGnJugWS1gPNwKci4ucDf4Ck\nW4FbAebNmzey1duEUVVaxKVzp3Dp3ClHtff0Bjsb29iyPxsiWxpa2H6glad3NLH62d309L4231dc\nWMDc6knZIJlWztwkUGZXT+LsKZOoKi3090RsTEszLAb7mzFwNv14fXYD8yLigKQ3AD+QdFFENB/V\nMWIlsBKyE9wjULNZv0yBmDctO3p426Kj13X19LK7qZ1tB4+w7UArOw62su1AK9sPtrL21UZaOrqP\n6l9RUsjZU0o5e0o2PGYnj+xyKTOrSinyhLuNYmmGRT0wN2d5DrDrOH3qJRUCk4GDkT1FqwMgItZJ\n2gJcAPh0JxsVijIF/UHy5oVHr4sIGlu72H6wlZ2NbexqamNnU/Z516E2nqk/xMEjnUdtUyCYWZUN\nk1lVpcyoKmFmVSkzq0qYWVnKzMnZQPFlUCxf0vyTtxZYKGkBsBO4GXjfgD6rgBXAr4GbgJ9GREiq\nIRsaPZLOBRYCW1Os1WzESGJqeTFTy4tZMuDQVp+2zh52HWrrD5NsoLSzq6mNF/Y089hLHceMTgDK\nizPMPCpMSplRWdL/PL2yhOnlJVRN8mEvG1mphUUyB3EHsIbsqbP3RcRGSXcBdRGxCvg68C+SNgMH\nyQYKwFuAuyR1Az3AbRFxMK1azc60ScWZ5HTdiuP2aenoZl9zO3ua29nX3MHe5nb2Nnew93A7+5rb\nWb+9ib3N7XTkTML3KcqIaeUlTK8sZlp5CdMqiqmpyD5PryhhWkUJ05PXU8uLfQjMhuUv5ZmNYRFB\nc1t3EiAd7G/JPg4c6WT/4eS5pYMDLZ00tHQcdXZXrillRUwrL6a6rJjq8mKqy4qoLitmSlnyum9d\nWRFTyoqZUlbkgBknRsWX8swsXZKYXFbE5LIiLphZOWTfiKClo5v9LZ0caOkLltfCZH9LB42tnew4\n2Moz9Z00tnYdN1wAKksLjwqQqeXZEKkuK2bypCImTyqialJh9rm0iKqkraSwwIfIxiCHhdkEIYnK\n0iIqS4tYML182P4RQWtnD42tnTS1dtHY2snBI6+9bmrt4uCRzv72LQ0tNLV2DTrXkqs4U0BVEiRV\npX2hUsTkAcuvvc62V5QWUlFS6LDJE4eFmQ1KEuUlhZSXFDKn+sS36+zupbm9i0NtXTS3ddHc3p3z\nuq+9u3+5qbWTbQeO9PfL/f7KYIoy2boqkkdlEiIVpUX9y+XFhVSUFlJZUtgfMn3L5cnr8uJC3xf+\nJDgszGxEFRcWML2ihOkVJ3+f9b7RTHN7NlByQ+ZIRzeHO7ppae+mJfe5o5sDRzrZdqC1f31bV88J\n/bzy4kx/IJYVZ5JHIeUlGSYVZZ/Lio9dl9tWXlLIpKJM/3uM15GPw8LMRo3c0cxZk4fvfzzdPb0c\n6eihpbMvVLo4PEjI9L1u7eysmkQnAAAHr0lEQVShtbObIx09NLV1saup7bW2zp4h524GKhCUFxdS\nNkjQTCrKMKk4Q2lRJnldwKSiZLk4aSvKUJrzuq+9r09pYUFerpjssDCzcacwU8DksgImlxWNyPt1\n9/TS2tVDa0cPRzq7aevs4Uh/yGTbWju6j+rT2tGTLHdzpLObxtZOdnf10NbVQ1tnL+1d2TAa5qjb\noIozBZQWFfQHyOvnTOEfbrlsRPb1eBwWZmbDKMwUUJUpoKp0ZMKnT0TQ1RO0dfXQ3tVDW2cSJsly\nexIs/W0569s6k/VdPcyeMmlE6xqMw8LMLE8kUVwoigsLmDxpZINopPlbNWZmNiyHhZmZDcthYWZm\nw3JYmJnZsBwWZmY2LIeFmZkNy2FhZmbDcliYmdmwxs3NjyQ1ANtO4y2mA/tHqJyxwvs8/k20/QXv\n88k6JyJqhus0bsLidEmqO5G7RY0n3ufxb6LtL3if0+LDUGZmNiyHhZmZDcth8ZqV+S4gD7zP499E\n21/wPqfCcxZmZjYsjyzMzGxYDgszMxvWhA8LScskbZK0WdKd+a5npEiaK+kRSS9I2ijpo0n7VEk/\nlvRy8lydtEvS3ye/h2ckXZ7fPTh1kjKS1kt6OFleIOmJZJ+/Lak4aS9Jljcn6+fns+5TJWmKpO9K\nejH5vK8a75+zpD9P/lw/J+lbkkrH2+cs6T5J+yQ9l9N20p+rpBVJ/5clrTjVeiZ0WEjKAPcA7wQW\nA7dIWpzfqkZMN/AXEfE64I3A7cm+3Qn8JCIWAj9JliH7O1iYPG4FvnLmSx4xHwVeyFn+G+CLyT43\nAh9K2j8ENEbE+cAXk35j0ZeBH0XEhcClZPd93H7OkmYDHwFqI+JiIAPczPj7nP8JWDag7aQ+V0lT\ngc8AVwJLgc/0BcxJi4gJ+wCuAtbkLH8S+GS+60ppXx8C3gFsAs5K2s4CNiWv7wVuyenf328sPYA5\nyV+itwMPAyL7zdbCgZ85sAa4KnldmPRTvvfhJPe3CnhlYN3j+XMGZgM7gKnJ5/YwcMN4/JyB+cBz\np/q5ArcA9+a0H9XvZB4TemTBa3/o+tQnbeNKMuy+DHgCmBkRuwGS5xlJt/Hyu/gS8AmgN1meBjRF\nRHeynLtf/fucrD+U9B9LzgUagG8kh96+Jqmccfw5R8RO4G+B7cBusp/bOsb359znZD/XEfu8J3pY\naJC2cXUusaQK4HvAn0VE81BdB2kbU78LSb8N7IuIdbnNg3SNE1g3VhQClwNfiYjLgCO8dmhiMGN+\nn5PDKMuBBcDZQDnZwzADjafPeTjH28cR2/eJHhb1wNyc5TnArjzVMuIkFZENim9GxL8mzXslnZWs\nPwvYl7SPh9/FNcCNkl4FHiB7KOpLwBRJhUmf3P3q3+dk/WTg4JkseATUA/UR8USy/F2y4TGeP+fr\ngFcioiEiuoB/Ba5mfH/OfU72cx2xz3uih8VaYGFyFkUx2UmyVXmuaURIEvB14IWI+ELOqlVA3xkR\nK8jOZfS1vz85q+KNwKG+4e5YERGfjIg5ETGf7Gf504j4A+AR4Kak28B97vtd3JT0H1P/44yIPcAO\nSYuSpt8Anmccf85kDz+9UVJZ8ue8b5/H7eec42Q/1zXA9ZKqkxHZ9Unbycv3BE6+H8BvAi8BW4C/\nync9I7hfbyI73HwG2JA8fpPssdqfAC8nz1OT/iJ7ZtgW4FmyZ5rkfT9OY/+vBR5OXp8LPAlsBh4E\nSpL20mR5c7L+3HzXfYr7ugSoSz7rHwDV4/1zBv478CLwHPAvQMl4+5yBb5Gdk+kiO0L40Kl8rsAH\nk33fDPzxqdbjy32YmdmwJvphKDMzOwEOCzMzG5bDwszMhuWwMDOzYTkszMxsWA4Ls4SkluR5vqT3\njfB7/9cBy78ayfc3S5vDwuxY84GTCovkCsZDOSosIuLqk6zJLK8cFmbH+izwZkkbkvsmZCR9XtLa\n5F4BHwaQdK2y9wy5n+wXoZD0A0nrknst3Jq0fRaYlLzfN5O2vlGMkvd+TtKzkt6b896P6rX7VHwz\n+bYykj4r6fmklr89478dm5AKh+9iNuHcCXw8In4bIPlH/1BEXCGpBPilpH9P+i4FLo6IV5LlD0bE\nQUmTgLWSvhcRd0q6IyKWDPKz3kP2G9iXAtOTbX6WrLsMuIjstXx+CVwj6Xng3cCFERGSpoz43psN\nwiMLs+FdT/a6OxvIXuZ9GtmbzAA8mRMUAB+R9DTwONkLuC1kaG8CvhURPRGxF3gMuCLnvesjopfs\n5VrmA81AO/A1Se8BWk9778xOgMPCbHgC/jQiliSPBRHRN7I40t9JupbsFVGviohLgfVkr0s03Hsf\nT0fO6x6yN/bpJjua+R7wLuBHJ7UnZqfIYWF2rMNAZc7yGuBPkku+I+mC5AZDA00me/vOVkkXkr2d\nbZ+uvu0H+Bnw3mRepAZ4C9mL3Q0quT/J5IhYDfwZ2UNYZqnznIXZsZ4BupPDSf9E9h7X84Gnkknm\nBrL/qx/oR8Btkp4he1vLx3PWrQSekfRUZC+b3uf7ZG8B+jTZqwR/IiL2JGEzmErgIUmlZEclf35q\nu2h2cnzVWTMzG5YPQ5mZ2bAcFmZmNiyHhZmZDcthYWZmw3JYmJnZsBwWZmY2LIeFmZkN6/8DvJNJ\nURko5lwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb921b38>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "J_history = []    # 存放代价函数值\n",
    "for i in range(N):\n",
    "    s = np.dot(X,W)\n",
    "    h = 1 / (1 + np.exp(-s))\n",
    "    J = -1 / m * np.sum(y * np.log(h) + (1 - y) * np.log(1-h))\n",
    "    J_history.append(J)\n",
    "    dW = 1 / m * np.dot(X.T, h - y)\n",
    "    W = W - lr * dW\n",
    "\n",
    "# 绘制代价函数J_history变化\n",
    "plt.plot(J_history)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('J(w)')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "通过J_history发现代价函数是逐渐减小的，在迭代次数接近1000的时候，$J(w)$已经取得了比较小的值了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后，我们来看一下逻辑回归的分类正确率："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "分类正确率：1.000000\n"
     ]
    }
   ],
   "source": [
    "s = np.dot(X, W)\n",
    "p = 1 / (1 + np.exp(-s))\n",
    "y_pred[p >= 0.5] = 1        # 预测正类\n",
    "y_pred[p < 0.5] = 0         # 预测负类\n",
    "accuracy = np.mean(y_pred == y)\n",
    "print('分类正确率：%2f' % accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可见逻辑回归分类算法，完全正确地将正负样本分开来了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. 总结"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "逻辑回归（Logistic Regression）与线性回归（Linear Regression）类似，但逻辑回归处理的是分类问题。利用了Sigmoid函数，根据极大似然原理构造了对应的代价函数。逻辑回归是一种最基本的分类算法之一。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
